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INTRODUCTION 

, This report is inteivSed.to provide a summary discussion of the Research 

for^ Better Schools Career Education Project during its first year of operation. 
' • » . % ' 

The topics included will be addressed from an evaluation peTspective. Oper- 
ational or developmental processes and issues will hot be discussed at length, 
bqt reference will be made* to their .inclusion in other existing documents. 
This report will cover evaluation activities concluded between September 1, 
197??and August 31, 1973. 

* 9 .' * . 

Program Goals ^ ^ * * \ 

The fdllowing have been stated as major Employer Based Career Education 

goala (FY 1974 OperaCihg Plans): • " ^ 

1. ^ First, the* project was designed to research thfe^question: to what 

extent arp employers willing and able to 'provide a total educational 
* experienqe for students? . " 

2.. The' second ma^or project goal involves an effort to develop a 

<j * ' ' , • 

o , 'transportable model of employer-b^sed car€|^er education for repli-^ 
*> . ' 

catixJh in othbr settings. 

3. The third major goal involves an overriding project.-commitment to 

. \ provide a sound educational experience for participatJ^ng students. 

Within. the content of these goals ^he program was operatioJialized employing 
three principal compone^s: Management Systems, Instructional Systems and 
Student'* Personnel Systems. ,The staff responsible for each of thea^ components 
develoj^ed the component-^p^cific go4ls, objectives, activities and procedures 
which operationally defined each component and, in sum, characterized the ^ \ 
pro^ject. These specific consdl^derations have been presented in the milestone 



Series related to each major 'component and may be' found in collected fprm in 
the "Operating Plane" (July 20, 1973)^, 

.The ini,tial project year goals ^ objectives were organized for evaTjr 
uation in*'Re?fprt 1.1.2 "Evaluation Model for Project Year One." For evaluation 
* purposes the Instructional- Systems Component was divided into Employer Systems 
' (including ^^veev Exploration, Career Specialization, and World of Work Seminar) 
and Instructional Systems (including the Individualized Learning Center, 
Supplementary Programs, Life Skills Programs' and Psychomotor Programs)*. This 
division was made in recognition of *the centrality of employer programs and 
the pi;x)bability that thesy would exhibit unique charactetistics. Although the 
evaluation structure was modiffed .over the course of the y^ar to reflect changes 

in project organization and conduct,- the resulting configuration of four eval- 

' / * ^ ' . ' . 

'uation components -(Employer , Instruqttottalj Student Personnel and Management) 

may be Used as a format for year-en^ r^poi;£ing. ' 

•■ ■ - ' ,u ■ 

The objectives ^©r each of these components, as ape£iried in the Eval- 
uation Model, were as follows: " ' . 

- ■ . • • . " ' " V ' ' / ■ 

1.0 Employ^ Systems Comi>^nent , * \ 

1.1 Employers will extend cooperation su(Cficient foi the provision 
of career exploration and specialization. 

1.2 Employers will develop in -the quaMty and "quantity of tfieir 

' ■ \ 

commltmentl 



1.3 Students will learn ^through their involvement in the' program. 

* * ' ' . ' * ^ 

1.4 *A World af Work Seminar can be developed and conducted with 

theri^objectlve of pproviding an overall view of exploration, 
• • * * 

information, skills, concerns and issues- related^ to career 

*' t . • ' 

* ^ . *, . . - 

^. ^pursuits, as well 'as a forH^i foxr<, student 'disclission of these 

'/ • ; " ■ . . , ; 

* matters. ^ • • 



Instructi6nal Systems Component \ * 

2.1 Instructional programs can>be developed and administered to 
meet . the' students* academic needs as defined by State' require- 

• ments, Academy requirements and student interests. *^ 

2.2 Psychomotor programs* c4n be developed and conducted to meet 
students' physical needs. 

2.3 A Life Skills Program tan be devised and conducted to promote 
^ the development of skills in interpersonal ^d^ationships , 

• • critical personal values and other dimensions^ -db-- be specified 
by staff and students! ^ ^ , • 

Student Personnel^'Systems Component 

3.1 A student recor^ system can b6 developed and maintained with 
the capability of resp.onding to staff and evaluation needs. 

5.2 Student Counselors (Developmental Advisors) will be available 
to advise students on career, academic and personal matters to 
the students' satisfaction as they progress tfirough the Academy. 

3.3 The Student Personnel System will* be able to adequately place 
students in careers, post secondary education ior another sec-, 
o^dary school upon their separation from the Academy. ' * 

^ ' .■ 

Management Systems -Component , 

4.1 The Academy facilities and resources *will be adequate for staff 
^nd student purposes. / * . 

4:2* -The Academy, administrative systems ^11 adequately serve staff 
and student needs. 

4.3 Students will develop positive attitude toward learning, 
positive self concept and positive attitude toward Academy 
experiences • * ) \ 



. These program objectives, ^as interpreted <nd stated by the. evaluation/ 
staff, formed the basis for the first year evaluation effort* Their genera^ty 
reflects the early stage of component development prevalent at the beginning 
of the first project year, DetailecJ^ standards were specified over time, as 
.will be described below.* 
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Evaluation Goals ^ ^ "* ^' r J 

•| The initial evaluation ^goals^ as statred^^ the Evaluation Model were: ' 

1.0 To develop,, e$tablis,h and maiijtain ^ feedback system which will 

collect ^^vP^^cess and disseminate information on all aspects oi 

^ ' ■ ^ - ^ ' ' ^ 

the project, , 

[ 1.1 This system will be designed to monitor student opinions and 

ft * * ^ « » 

concerns, refer them to^^appropriate project ^taff and relate 

] ' results to students. 

^' . ^ . . • 

/ 1.2 It will, be designed^ to monitor staff opinions and concerns, 
define suggested changes and^^disseminate results.* 
; fJ^^3 It will be designed^ t\r mbnitor employer opinions and"* concerns , 
refer them* jto appropriate staff and relate results to employers 
1.'4 The system will /also provide symmary data for progress and 
^ ^ trend analysis during the course of project year one. 
,1.5 This'informairion bank. will further be a resource for staff 



working on Individual student or group .problems • 



/ 



2.0 |;o^ define, -measure, analyze and interpret project effects|on students 

*2.i; Thi^ effort is aimed at evaluating the feasibility of employer- 
based systems. " . • 



. . 2.2 'It will further evaluate the' effects /principally on students) 

* * 

of the project systems in relation to 'staff Expectations and ^ 
nat'ional or lo^al norma. * ' ^ ^ 

The evaluatidn, system was designed to be developing; adap^ve andV^esponsive, 
During the first, project yW.its principal function was -Wen to be that of ' 
a formative ^gent. The- data generated in this pursuit would'have 'aurnmative 
. Unplicatiop. but the state of the art. ' especially with r.egard to instrumen- 
' tation. would not allow a more .than rudimentary summative effort'. It was-' 
. Planned that Instrument development undertaken during the;first year and.' 
refinement of evaluation technique would progre^ively enable' more' definitdve. 
useful and sophisticated evaluation of 'the Research fo/^etter Schools mode'lrr. 

VEvaluatlon Model ' ' ' ' . * • . 



^he evaluation^model thus projected an evaluative pr'ocess which'woul^ 
initially focus on formktive issues and gradually^evelop summative- capability 

. The measuren^nt p'rocess' would further .of ne«^^sity. be initially subjectively' 
based and wo^ld proceed toward objectif ication 'as refinement wa; possible.. 

. It vas recognized that tfte lack of both derivativ^e operational definitions'' 
with respect to program objectives an^ appropriate prepackaged tnstriimentafci^n 
would be serious impediments. Ittas'^plannfed that . the developmental fois L 
the evaluation effort would be in* both the;e area^^. while the forLiv^wl 
uation would ^im to assess identified elements of program operat ions 'anl^ro^ ' 
vide feedback for program improvement." It was, planned that formative resi't's 
would also providA basis' for summative evaluation. Both formative and -sum- 

: ' • ( ' • • • 

native efforts would draw upon' the^ following sourc^f information: " ' ' . 
!• » Student questionnaires and interviews 



6.- 



2. 
3. 
4. 

5** 



Staffft estionnaires and interviews 
Enfployer questionnaires and interviews 



^ '8. 



Student background information 

Staff background information . * 

6. Employer background information 

7. Measures of student progress 

" . •* ^ ^ - ' ^ 

Observed events measured against objectives 
9. 'Observed events measured ag^ainst established regulatory criteria 
10* . Observed , events measured against rational subjective, criteria 

The 'instruments (except standa^ized tests) and informational' formats involved 
all had io be developed -during the course of , the yeari~^*^detaSed d<»scription 



-1 



'i : 



of this process may be found in Milestones lOj^ll and l|g*!Evaiiui^^^ of the 



Operations of the Academy for Career Education*' (June r)^f l97^'^i The data '' '^f^y-' 
^us gathered were analyzed and presented in a series Slrepdi^tifdesfgneci 



to provj.de £n.fonna.tion for program development .and ?t6.,rt^estiglf^ 

and efficacy issue's as,, tKey rfeiate to program goall. '^WM of over •"^■^I'^x 



^if ty major reports is outlined in Appendix A. 



Report F ocus . - • - * 

For FY 1573 the e^^luation processes and prod^cts'tl^^^ confined ^^-'W' 

. ^ ' ' ' ' f 

to the operational aspects of the Research f or 'Bet|^r^J|g|^^^ Educafioi^ J> 
Program. Such ^reas ^s fiscal analysis, replicablt^ifpSid'^iiliterm planning! t 



were not considered to be manageable elements for Uif^Mti^^^he first 
year evaluation. The present report will reflect ^Kl^^^^^pn. During 
FY 1,973 these elements were a principal concerif o 



nagement 



and extensive discussion of them may be-'found 
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f 



(July^20j, 1973). The present report is' designed to serve as 'an evaluation of 

/ * 

the first year of /Academy for Career Education operations, based on the 
Evaluation Model (1.1.2) and the resi/lts obtained through its conduct. 
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' METHODOLOGY ^ - 

' . % ^ - ' ' 

Subjects ^ \ - • ' , , . 

' First-year Academy studeijts-were recruited from the eleventh gradeT'^' . 
populations of Philadelphia's publTc (25) and parochial (13) secondary . ^ 

schools/ These ag^school^ pombined enroll approximately 50,000' studTents^ 
in each grade. Newspaper advertising and mailings to school principals and 
guidance counselars were the media used for information dissemination. The 
recruiting process, which was conducted mainly during May and June 1972, 
resulted in 277. applicants. Of this number, 216 primary applicants' were 
selected by a lottery procedure, stratified to achieve an equal sex iSlx and 
a 8amj>le representative of the city's geographical distribution of -secondary 
students. These applicants were all screened for disabling ^psychological, 

* . ' ^1 

•learning or somatic problems. Candidates w^re also required to have sue- ^ 



cessfully completed their academic studies through the tenth grade curricu- ^ 
lum. Of the 216- students ^Selected by the lottery^ 119 were both, interested 
in actually enrolling In the Academy and deemed acc^jgtable by the screening v 
.staff.' Another 12 applicants' withdrew before >,the'.start of classes, leaving* 
107 students as t;he final s^ple for the first year. Insufficient applicants ' 
for a randomly constituted stratified comparisjon group and logistical '•dif- 
ficulties In- constructing a city-wide group of matched' subjects rendered 
control gro^Ss not possible for the first "year. ,Thus the total number ofi ' 
subjects involved in the study was 10?! ^ ^ : 

The student group had the following characteristics (for a complete 
discussion see Evaluation Report 5.1.2 •'Characteristics of the Acadeiny Student 

* ' » ' 

Pbpulation") : v 



V 1 ' 41. 



9. 



4 . 



• the majority of stydejits were 16 years of age uppti enrollment, in ^the 
Academy* * ' . " ' • , • ^ * . • 



0 V 
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,Nuirci':R OF 


. PKRCKIJT'OF 


15 years 




■■• -33.r-' . 


" lb y.c ir.^ 






17 yeayr> 






. ■ ■ 
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r - ' - ■ 


^fOTALS, 


107' ■ ' , 





••a 



r - i 



■ . 4 



- The tacial\split,was approximately 60% Black and '40%. White.' .The seV- 
. split,, wa§ approximately eyen. The Black Female subgroup was! disprot>Drtionately ♦ 
-large, while the. White -^'emale suH^roup was- disproportionately jma^ ^ - ^, 
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"NUMBEIJ^S .OF "Sfi'DEIJTS 




Race 






w . Total" 


» 


Bliick 


■■■ 26 ' 


39' 






White 


0 

27 




■ h 




Other - 




: 0 . 


•'1 




.Total * 


"■54' 

4 " * 


• 33 




■' ao7- •■ t 


\ 
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Percent of Student<5 


Race ^"^"^^^^ 




Female 


Totnl 


Black 


24,3 , 


36.5 


60.8, 


VThitq . 


25.2 




38.3 


Other 




o" 


.9 . 




30.4 


A9 .6 


100.0 ' ' 
* 



A l*ge majority of students previVtusly attended the public schools.. 



TYPE OF 
SCHOO], . . ■ 


Ni:yj:i:R OF 
S'rrDi,::!:; 


Pr.RCENT OF 
STUUKN'TS 


1. Public ' 


78 


72.9 ' 


2. Parochial 


■ 23. 


21.5 


3.* Private 


-• 2 • i 




4. Other 




- 3.7 


TOTALS : 


107 ' 


100.0 



Most students had average grades of D or lower ^f rom their sending 



schools. Only 18.7% had a B or 'higher, and 19.6% had a C average. This * . 



is not a refined measure,- and grading systems differ firotn' school to school," 
but these statistics do suggest that Acadpmy students had generally low ''f' 
levels of performance in their previous schools.^ , V f / - 



n. 



5^' i. 



■F or' 59 & hel6(^: 



Unknown 



TOTALS: 



V' W/t^^^v.-^- 



23. 



2'4 



10-. 



. , 19-. 6%. 



21.5% ' 



107 



;ioo.o% ,; 



Complete tod consistent data" on past standardized test scores were not 
available, but partial data suggest depressed performance in this area 
also. A majority , of stuiients placed below the 30 tli percentile. 
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. - STANDARDIZKp TEST 
" PERCENTILES : 

' , - ' / 


NUMBER OF 

STUDENTS • . 

/ 


- percent' of 
students; "5*; ■ 




■" / •■ 

• .- 9^-90 ' . - 


. /' 
•4' 


•" '8.7% A 




89- 80 , ^ 


r — : : — — 


1 

10.9% 




: , ,/79-,70 


\ /'!.". 

r 






69-60 - ' 

/ 


' ' 2 


'■V// 






^ ,'59-50 


■1-/ 


;;'?■(; 






' /49-40: '■ 




8.7%"-'^ ■ '.fJf 




. 39-30 ^ • , 


/3 




. \ Ij 


\ 


' • / 29-20 


' ' '7 




y ■ 15^% , ■ 


i 

■ ^ 


,19-10 ;■ 


'•. 6/ • 


' ■ /^13.0%.-V<^ 

/ ■■ ■■'.■> 




>■ / »-o ■ / 


.' ' 13 ■ 

t 


/ * ,28.. 3% ■ Jr^i^ 


V . ' ' 
'^-^ 


' lOTALS: ' , 
*5 « ! —4 




■ 100.0% 


- Ay- 
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• rt 
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■11/ ^ • 

,• students '^rfepo;rted the .following 
•'frequency. 'for mothers ^unemployed (housewife)-, 



-'• ^- /W "C-v '.•"/ • J'V'Tj^ yvs- •> :. • / / 

L7 ' 'JJ'/ 'J ^^^yf^i^^^t^r'i^tktttyL oc^^p^t'ion.stui 

/;/ca^«g95E:ie5/i«i;th^'^ 

V'f'^^^H/^^^'s^^^.'^^/^aJ^ Those ,m6st:^ftequently,Wp6rted foi? fathers were: 
/ 7 • -?i5aft§mfeh^6peratives'; professional and sefviceA ? '.The 'high'' nUber of rf^^^ /' 
? ' rr'.^' tespoQdents for-' fathers* occupation is pr6^?bly/pkrti^ll'y accounted by - 
V'^::-..; ;:--?.eparated^p:arents; ■• „ , ; .;, /'---^ / ' .■ ;' •/ 



■ s ■ 



* ^ • • ' '« * 






■ ■ . :v'^ 




• . ' • • ^ ' ' 
/ * JOB .C^CTEJGOrlY v / 


. ' MOTHERS' 
• OCfCUPATION'S^ ■ 


' FATHERS^ 
;'0CcVpATI0NS 




" . 01 ,Pf 6 f es^^iT^l/;! , / - _ 








.02 .Jechhical.,. / • 


\ lb -.' '. 




V 


OS'. -/Fjamers > ; yV-V 








Of Iferi^g^^^ ^ ' 




■ / : 7 - 


^ 1 y * • 
^* * 


-05 .CJer JLc^l/ - ^ ' ;/ 




-2 ■ 




06 Sale^. ■/ \ 








07< /C^i/af^s^iien . J 








08 >'.For,enieil* \ , 

' . . • . '/ 'S . ' ' • . ' 
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• 09" /Operative^ . ' 

^ ^ ' ■ \ 


6" 

/ 
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/ • / ^ / , f - , ^ 7 / / , . 

11/ SetvlCG'^/ 


3 ^' 


8 




n Farfn-Lcib'of ' ^; ' 

i' ^ r f ^ • . . •/ 
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1- - 3 
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1 




j^"! 99 '- Unknown^ \^ -y-^ y' 




. 48-> 








107 


• /-'ii* 
= • --i-'/'-r---- ' - 
. -cfe- 


^' ' ' ' ' ">'^' t 

r t * > . 

► . » / 






ERIC 


.*;■■ .N 







, ■ v,- / -.^r . . • '^r • - ' 

- I f . \ J," ■ , . ■, • , . , ? ' 

••>'- ' - ■ '•■ •■. - ' ' /■■.'■'/ 

• ■/ ^ ■ , - The.Mmons- ana Amions /^^^^^ (qt) verbal Vercyptual intelligence 

" ;• scote-i? 96/3|.i ,Kith a r^nge of 56.to 12.7^/*^This indi^c^d a 

.^^ility measu^e'd by this .IQ test),vith te^^oi^lV T^igl^, 
/ U.'-' the distribution, jf -obtained se^&^^^^^^ 

. ' ^ ^■■■•iniV^^ fiiom'a npnning sample also.'.lndicat^d a ^latively "not^nal,"- though" " 
y/ /^Aoniewh^t /lQij\ prbfi-re-of sc6tes-.-_ /•. . ~ "<.- 

/ ; /In. summary the group of subjects Varticipating in* the^first'-ye^rrof , / 

: t «icadeiny bpei;atigri5 'numbered 107. A majority were 16 year-old llth^graders - 
who had previOAisly attenle'S public schools;throughout Philadelphia.* There 
/ ' , approximately the same number of Males?aii^ Females, while Blacks out- 

numbered Whites by a- margin of 3 to ^2. Most sttidents had performed poorly 
/■•■ / in their sendip^ schools, -but ha/' :lose to average IQ scoi;es. This suggests 
. -.ythat observed low achievement may-b^ perfopnance' effect rather than a" 
C^^li^omenon related to.ability. Reported parental ■occupatidn •types were 
'r,.s^t64d through most of the categories with many "Unen^loyed (housewife) 
, rAnd^Unkrfovp;' Responses making generalization difficult.' 
■■ ' grodjps.pf project participants whicTi must be considered subjects 

oJ..the atudy inr.a aifferent sense were emplov^rs, staff .and parents, 

.".^ Empi-d^^^s were ;ii,olicited throughout^Jl)4 Philadelphia area by rejie^en- 
tatfvep of=-t"he' Greater ;:^hiladelphia Chamber of Commerce and Research for 
Better '^choois. VOf the^ i^roxlmatel^ 70 companies approached for partiqlpa- 

tion in the AcaVieny progfkiaV 46 agreed. Potential employers" were initially- 
' ^ , ~,, \ - ■ '■ • ■ ■ . ' \ ■ 

.. • selected on an libitum basis,, but in t;ime vacancies in the cluster ' • 

str.ucture determined what type of firm should be approached. The ^'ployers 
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Ifcb ' ■ ' P^'^^^cipating.^dui^ng'tK^iiyst yMr/are listed by number beltiw 
'''selected charact^-fecist'f^'^t, ^ ' *! 



along with 

charact^-feijist'f^^i 



Eaplbver Type of Comp ly 



1 
2 
3 

. 4 
5. 

7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
'17 
18 
19 
20 
21 
22 

24, 

25 

26 

Ut 
28 

29 

30 

31 

-32 

33 

34. 

35" 

36 

37 

38 
\39 

40 

41 



/ 



44 • 
45 

46' V' 



retail food 

automatic merchandising 
petroleum products 
. investment brokers. 
telephoUfe conmunicatlons 
investment brokers ^ * ^ 
banking ' , ' 

travel agency 

electrical apparatus manufacture 

space re-entry systems 

advertising 

hospital 

government 

travel agency i ' 

broadcasting 

legal services 

business representative 

gas products service 

water and sewage service 

banking 

educational research 
synthetics taanufacture 
diversified retailer 
^research & d€velo*paent • 
Igovemacnt >^ 

- gQvefnaent 
travel agency 
diversified retailer 
goyemaent 
broadcasting 
hostelry 

hostelry ^-^-9 
investment brokers 
'dvess manufacture 
Insurance 

- clothing manufacture 
hostelry 

. library , • * 

? publle transportation 

pharmaceuticals manufacture 

clothing manuf'acture 
government 

clothing manufacture A 
publisher 
Mg^zlne 



Ehiployees 
2,000 

5, (/do 

2,000 
75 
8,300 
230 
1,500 
10 
3,300 
6,000 
200 
3,300 
2,000 
•23 
200 
200 
. 60 : 
3,000 
1,600 
. 750 
175 
1,200 . 
/'350 
' 100 
160 
100 
1 , 
3»50Q 
1,350 
27 . 
450 ' 
400 
100 
400 
4,520 
2,000 
900 . 
860 
6^^800 
950 
400 
4O0 
6,000 «^ 

960 
1,300 
30 




Profit or 
r NonrProfit 

profit 
ptof it ' 
profit 
profit 
profit ' 
profit 
^ profit 
profit 
profit 
profit 
profit 
'* non-profit 
non-profit 
profit ' 
profit 
profit; 
non-profit 
non-profit 
non-profit 

profit 
non-profit 
profit 
profit 
non-profit 
non-profit * 
. non-profit 

•profit 
^ profit 
non-profit 
profit • 
profit' 
profit* 
profit 
profit 
profit* 
prolix 
profit • 
non-profit' . 
non-pifofij: .1, 
. profit 
profit 
pro£it 
•non-profit 
profit 
-profit 
profit 



As can be seen. the employer sample represei^ted a wide variety of 
products^- and^services. Of the 46 firms, 13 (28Z) were non-profit a^d^the 



. remainder were established as profit-making concerns. The companies ranged' 
i, in size f.rom 1 "employee to 8,300 employees in the Philadelph^Ta area. The 
-^average: size was 2,600 employee^,- with 22 under 500 a;pd only 7 under, 100. 

^ Eighteen (18) of the employers operated only in the P^ladelphia region, 
while the/remainder covered a larger area^. The total size in. terms of em- 
ployees .nationwide of the latter ranged from 36 to 1.200.000 with an average 
of 125,000. Twenty-four (24) of the firms were experienced in educational 
or. training programs for employees, and 22 had been involved'in other programs 
for secondary scho^^l students. A small number of employers (8) were re- 
munerated for their participation; while the large majority were volunteer."^ 
In summary,. the employers involved in the program were a ' diverse ^sample . 
from the Philadelphia business community (including government) . The firms 
or agencies tended to be large operations wi^tR most Ti'aving organizati'ons whch 
exceeded, the Phila'Helphia area in scope. Approximately half had previous 
experience with educational- or training programs. . ■ ^ 

With regard to staff, Research for Better Schools 'persoiiel were 
allocated to operational ^functions as follows^: 

1. Management Systems Component - T 

2. Employer Systems Cojnponent - 4 * 

3. Instructio^l Systems Component - 4 

'4. Student Personnel Systems Component ^- 4 ' 

I 

An additional 13 positions. were allocated for the development, administrative 
and evaluation staffs. Background ^br characteristics , data were not collected 
for*staff members. 

No systematic data were collected^ o^ the parents Of Academy students. 
. An effort in this regard was. intended, but time did not permit its inclusLon. 
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Treatments \v 

The treatment ap^ied to students consisted mainly of the Academy 
curriculum. Element^, of this will be only briefly outlined in the present 
report-; for a detailed discussion see the Operating Plans (July *20, 1973) . 

t 

Instructional activities' were conducted by three of the components identified 
above: Employer Systems , Instructional Systems and Student Personi^el Systems, 

> 

Uaming activities within the Employer Systems ^:omponent. were all 
conducted -at participating employer sites and may be classified as Career 
Explorations,' Career Specialisations and World of Work Seminars. The first 
two activities 'involved either individuals or small groups being exposed to 
programs developed by the employers and model staff and designed to provide " 
first hand experience in a particular job, company dr industry.. These ac- ^ 
tivities generally were, scheduled f St. four dne-day sessions' ovet -the couie • 
of a month, with three related employers, a cluster pi^DvidiJg instruction** 
for eajh academic quarter. In all, twelve clusters of empL^rs. of f ered ac- 
tivities during the first year; they .were categor^ed as follows-. Communi- 
cations, Finance, Government, Health, Logistics, Manufacturing, Research,' - 
Sales, Systems, Utilities, Hotels and Motels, and Apparel. The World of 
Work Seminars were designed and conducted by the Chamber of Commerce with ' 
the aim of providi^ an overview of the area industrial base and the career ' 
opportunities generated thereby; as well as providing a vehicle for stud^t 
-interaction .concerning the'ir employer-based experiences In ^eral, the 
activities offered under Employer Systems we^e^^ended t^^develop career 
awareness through direct experience with rj^esentative employers In the 
Philadelphia community. - . , ' ' \ 

. Under Instructional Systems students were offered the Individualized 
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Learning Center. Electives and Physical Education.^ The basic purpose of the 
" Individuali2ed Learning Center"wa8 to fatllitate the attainment of minimum 
performance levels '(approximately 9''th grade) in Mathematics and Communica- 
tions Skills for all Academy "students . This was. to be accomplished through, 
the direct provisit^n of individualized 'instruction using commercial materials 
indigenous to the Center, well as- the coordination of pertinent learning ^- 
activities which could completed at other locations within -the program. • 
Students were assigned Sleekly time periods in the Center and placed in cur- 
. riGular materials baaed on their needs as determine'd by diagnostic tests. 
The materials includ^ed Individualized Learning for Adults. EDL Audio-Visual 
Baterials, programmed grammar texts, advanced mathematics courses and others. 
Elective' activities were'- .designed to pro'vide" academic . instruction comple- • 
mentary^and supplementary to thl basic educational experiences available 
. through required course offerings. A wide variety of activities which would 
meet* students ne^ds and interests, were intended. Instructors were solicited, 
mostly on a volunteer basis, from'^he Research for Betteif Schools staff, 
participating employers, and other agencies and individuals. Physical edu- 
cation activities were designed toN^^ovide students with the required mini- 
mum of two hours per week of physical ac^tivities matched to their ^eeds and 
interests. A wide variet'y 6f activitTit were pursued through area YM(W)CA 
and YM(W)HA organizations, neighborhood recreation centers and programs 
constructed by Academy ^taff. * ' 



1. 



Life SJcills programs' were initially .included here, but were transferred 
to Student Personnel Systems during the second quartej of operations: ' 
for this reason- they are discussed under the lattJfer.iiladl,ng irt the present 
report • - , ' 

^ . • . \ . ■ ' 



. Under Student Personnel Systems students were offered Life Skills ' ' - ' 
^ Exploration and Specialization. Life Skills Exploratio^ initially took the- ' 
form of Workshops in Living which eventually evolved into Group. Guidance 
Sessions. These activities were fociised on developing self-knowledge and 
skills in -.interpersonal relationships. As they." evolved , the concept o§ group 
•.guidance as a^viable form of career guidance assumed more primacy. Students ' 
were, exposed to generalized career information, career. seeking skills rfnd 
career-related-self appraisals. These sessions were pegalarly schedu/ed * 
small group activities. Life Skills*Specializations were design^^' to expose 
students, to real life social and psychological issues through oaUite ex- < 1 
_perience With community s,ervice agencies and groups. These activities were " , 
.elective. Of instructional valnie", but in^ an 'entirely informal sense, 'students . 
were also offered extensive individual counseling within the Student Personnel 
Systems Component. - ' " 

;- rn siimmary, students were generali^ scheduled for the following weekly- 
activities as a requirement of the programi Career Exploration for 6 hour's. 
World of Work Seminars for 1.5 hours. Individualized Learning Cent^lr for 4 
hours. Physical Education ^ol: 2 hours and Guidance Groups for 1.5 hours.. In ' 
addition,, stpdents needed 7 hoUrs of electives to-be selected iE'rom the follow- 
ing offerings: Career Exploration and Specialization, Electives and Life'" ' 

* • • J? ' ^ ' . • ■ 

Skills Specialization. Students wereithu^ scheduled for a program of approx- 
imately 22 hours per week. ' ' 



No- treatment in the tradttional sense of t;he word was applied to employ- 
ers, staff or parents. Their interaction with^the Academy activities and 
,ft,udents defined the nature and extent of their experience with the exper- 
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:<oJ>imental,program. In the case^ of employers ^an orientation to the program 
- Ka^ ^provided by project staff agsistanL was given 'in the creation and 
,- condiict of their learning ^ctivitie^,' and continual liaison was provided 

by the Field Coordinators. Each of these three staffs* members was assigned 
• approximately 10 employers as ^ "case load." 'since only 10 to 15 different 
employer .learning activities wfete in progress .during any one lea/ning cycl 
the^amount of attention which could be devoted by the chrefe Field' Coordinatdi-s' 
to eafch employer was -consid^^bU . - Meetings where numerous employer repre- 
jentatives, would interact and share experiences were also planned. 

Staff w^re oriented to. the project and their responsibilities by the 
. ^' supervisors in each component area . Staff sessions in the form of retreats 
^ - and othet meetings/ were also planned. ' . 

■• Parerits were oriented to the project inVouP sessions conducted by the * 
• \IJojecf Dir^c,tor and. individual interview sessions .With project staff mem- 
bers._ Meetings Were also planned during the course of the year. 
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lostruments 

fhree standardized -instrument>v«M;e/selected for adihlAistration to 
Academy students. The Comprehensive Tfests of Baslr Sk^^^« ^..o ^sed to 
measure perf oiihance i^ traditional achievement areas (Level -3 Form pretest i 
Level 4 Form Q posttest).. The Test of . Economic Understandlnf > was selected 
•to measure knowledge of basic economic concepts (Fonn B pretest, Form A 

The Personnal Orientatio n Inventory was used ttf- measure af- - 
fective dimensions (1 form):. Th^se instruments are not appended, but are 
fu^^ther described in the presentation of results below. 



The following instruments were develpp^ by the evaluation staff for 



use with students, and are found in Append iil B in their final form'. A 
^Student Information Sheet" was cp^pleted ^or each student from i'^mation 

Ls provided basic background and demographic' 
,infomation on the student s^ple. A "Career E^loration^^estionnaire 
. '.developed to measure stud^t, attitude^fend progress relat^fe Ab their Catle 
/Exploration experience. A questionnaire, the "Student Evaluation Report." 
.was develcJ^ed to monitor student att^.61^ and opinions concerning all 'aspects 
of the Academy program; a^"Student Evaluation Interview Schedule" was also • 
^d4veloped to -gather similar .information in a personal interview format. A- 
I'Student Questionnaire" was constructed to. measure: Is tudent expectations' of 
the Academy, reasons for' enrolling and general cjjinions.' Each .of thes^!^% ' 
strum6nts was refined and- revised' o^er the courses of" the year, based up^'n'the ' 
\quality of item performance an4; the content of program revision. 

Thi following^instruments were developed- by the evaluation staff for use 
with;:he employers, and are found in Appendix B in their final foml. The 
"Emp.loyer Information Sheet"' was completed -for each participating employer 
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by telephone, interview. This provided basic data on the' employers invblved 

in the program. A questionnaire, the "Employer Evaluatidn Report." was 
■ ■ i ' 

developed to monitor employer ' attitudes "and opinions concerning the Academy 

program and their participation in it; an "Employer Evaluation Interview ^ 
Schedule" was also developed to gather simil^ information in a personal 
interview format. Each of these instruments was refined and revised over 
■the course of the year based upon thi quality of item performance and the 
content of program. revision. • ' 

Only one instrument, the "Staff Evaluation Report," was developed for 
use with project staff members. This instrument was designed"^ to elicit 
staf^ attitudes and opinions concerning various aspects of the progr&m, as 
well as to, gather -basic. data oh opefrational functioning. This instrument 
was also refined and revised over the cours^e of the year: 
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The Academy program was offered "to participating' students on a quarterly 
schedule. Each learning activity was designed tq encompasfe 12 veeks . In- 
dividual student programs were fcpnstructed by stydents' and counselors based ' 
on expressed interest and evidenced academic nee^d. The generalized require- 
ments consisted of 1 Career Exploration Cluster per quarter, 4 hours of - 
Individualized Learning ' Center activities per week, '2 hours per week of 
Physical Education, the World .of Work Seminars and the Group Counseling 
sessions (both -1.5 hours). In addition, elecayes were available in Career 
•Exploration and Specialization, the Electives (Supplementary) Program,' and 
Life Skills Specialization. Each student was scheduled for approximately 
22 instructional hours per week. The specific leamflt actl^rities available 



are listed below; numbers in>arentheses indicate th^Sarter of initiation- 

^ . • ? \ %. : 

the 400, 500 i^nd 900' Levels constituted the ElectiveS Program: 



lOq Level - ILC En||I<8h 

(1) 101 ^--^-Basi!c'%^J 

' (1) 102 lith Grade 

(1) 1Q3 Independent Study 

(3) 104 - English Group 

200 Level - ILC Math * 
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(1^ 2(fl - General Math 
(1) 202; - Plane Geometry 
(l)-,203jw- Algebra 1 
(1), 204f- Algebra 2 
(1) i05'~ Trigonometry \ 
(1) 206 - Independent Stu^y 
(3) '207 - Math Group 

300 Leve^V Career Exploration 

(1) 301 - World of Work Seminar 

(1) ,302 - Sears 

(1) 303 - Wanam^ers 

(1) 304 -, IRS ^ 




300 Level (continued) 



.(1).305 - Hahnemann - 
U) 306'- jPGW - 

(1> 307 - GE/RESD . ' / 

• (1) 308 - Rfis ■ . , r 

(1) 309 - WFLN ^ >' • 

(1) 310 - GE/Swltchgeat " ' 

(1/ 311 - 'Udntlnental Bank < 

(1) 312 - Acme . 

(1) 313 -• Crestravel 

(1) 314 - Marine Corps Supply 

(1) 315 - Bell Telephone* Co. - . ' . . 

(1) 316 - Pepper,, Hamilton & Scheetz 

(1) 317 - University Cljty Science Center 

(1) 318 - Grw and Rogers ' 

(1) 319 - Rohm and liaas ^ ' " 

(1) 320 - Butcher & SKerr^rd 



(1) 

^1) 
(1) 
(1) 
(1) 
(Da 
(1) 
(1) 
(1) 
(1) . 
,(2) 
(2) 
C2) 
(2) 
(2) 
(2) 

(3) 
(3) 
(3) 
(?) 

(i) 

(A) 



321 in;^ ■■ " , . 

322 - Civil ServiM Commission 

323 - ARA^ . * 

324 - Philadelphiaf Public Library 
323 ,- ARCO - ' ,'' ^ 

326 - KYW 

327 - Wyeth - . 

328 - SEPTA 

•32^ f>h'iladelpliia (Jater Works 
330^ Cinderella Dresses 

331 - Marriott 'Hotel & Restrtupant 

332 H^PSFS (cpurse)- 

333 - BQche & Company (qoarse)' ; 

334 ,- Ben Franklin Hotel » 

335 - Bellevue Stratford Hot^l 

336 - -Burnham *- , * 

337 - J. H. Cohen 
33f - Holiday -Inn 

339 - Robert Bruce Co. - - 

340 -. Yarnell • , " 

341 - Soowal 
342^- HEW 

343 - Chilton & Co.. 

344 - Philadelphia Magazine" 



400 



Level - Seminars >& Courses /English/ Liberal kft^ 



(1) 401 - Llterature^as 'Self Expression. 

(1) 402 - Continuing English Improvement 

(1),403 - Applied Imagination' <^ • 

(1), 404 - Creativity • , ^ 



400 Level (continued) ' , ^ • 

(1) 405 - Guitar 
*(1) 406 - Art & Design 
J (1) 407 - Intro to "Law, Law I 
(1) 408 - Afro-American History 
(1) 409 - Travel 

(1) 410 - French " * \ • 

(1) 4H - Spanish 

(1) 412 - j,German. 

(2) 415 - Art & Communication 
(2) 416 - Law 11^ 

(2) 417 - MoQd Modifiers' 

(2) 418 - Communications - 

(2) 419 - Short Story " - * . 

(2) 420 - Self Exploration through Writing 

(2) 421 - Creative Photography 

(2) 422 - Social Anthropdlogy 

(2) 423 - Camping, Hiking & Wildlife Management 
(2) 424 - Films 

rll " Silkscreening, Printing & Blockprinting 
(2) 427 - Drawing & Painting 

(2) 428 - You and the Author r 

(2) 429 - Fundamentals of American Law ^ 

(3) 430 - Cultural Anthropblogy 

(3) 431 - What Makes Sammy Run? ' ' 

(3)^432 - .Literature of the Civil War ^ 

(3) 433 - Basic Drawing 

^ (3) 434 - Introduction to Architecture 

(4) 435 - Archaeology . . 



500 Level - Seminars & Courses/Math, Sciences, Business 



(1) &0i - Shorthand I 
{!) 502. - Typing 

(l)i503 - Secretarial Procedures 

(1) 504 Stahstics 

(1) 505 - IVaychology I 

(1) 506 - Chemistry I ^ 

(1) 507 Pljys'ics I • 

(1) 508 - Coiiq>uter Science 

(2) *509 - Psyfchology II 

(2) 510 - Personal Grooming & Professional Modeling 

(2) 511 - Heart & Lungsk^ 

(2) 514 - Muscles & Movement ' 

(2)^513 - Eating, Using Excreting 

12)^ 514 - Procreation 

V:\ ^ Health Concerns of the High Scl^ool Student 
" Records Management & Office Procedures 
(2) 517 - Business Administration - 
(2) 518 - Early Childhood Development 



500 Level (continued) 

III ~ ^^yP"nch Operation T,. • 

(2) 520 - Sewing l"*^' 

■^mJ ~ ^^"^"8 & Crocheting 
3 III ' ' Conditioning 

3 III ' f^'^''^ ^"""^ Together 
3 lit ~ u^f ^ ^^"8le Student 

(3) 525 - Weight Control 

fx s^7 'J*P^°"ti°"°f Mathematics 
W 527 ^ Getting Your Body Together 

^ Level - Career Specialization ' . 

(2) 611 - SecEetiria'C • Vi'-r ". . ' T"^ 

^.^i-W-'^OZ^-'Auto' Mec^^i^ 
^ . 1* (3)_ 603 - Communicfttiona 

' ^ \ - Boutique (Sales or Apparel) 

c PJ - Crestravel (Logistics) * ' « 

VV' (3) 606 - Rohm & Haas (Research) Z 

(3) 607 Psychology 

; : (3) 6P8 - Health ■ ^ • - ' 

(3) 609 - Logistics ■ . " 

\ " (3) 610 - Law 

• . V ■ - ■ I ' 

. , , . ; 700 Level . Life Skills Explorati6n\and Specialization 

' f (1) 70r - Workshops in Living 

' (2) 702 - Grouj) Counseling. Sesisions 
^ ; (3) 703 - Ecology Food pS-op ' 
(3) -704 - Police Department . 

/^n?? ^'^y WX«e. Center " 

(3; 706 - Pa. School for .the •', 

• • )Z l^'' ~ Center/for' Older teftple 

(3) 708 - Dreuding Nuraing Home 

(3) 709 - Nationalitig^^ Service" Center 
CA) 710 - Planned Parenthood 

ul ~ Sf'^^^'^^*" Street YMCA 

//( l}l ~ Convention Tourist Bureau 

(A). 713 - Wfiittier Child Care *. 

(4) 714 - Model CUies • !% ' > " ' 

. (4) 715 - Allegheny>Housei I' ' : 

W 716 - Court Ball BondBrogram ' ' ' ' ; 

(4) 717 - Community Advancement Program l 



800 Le^^ei->-\.i>hy8ical Education': !■' 



\ 



/ sip- 80i r^elf . Delen's^ ' - ' 
. •Cl)_.802 - Gyiimaatics 
,/ (I) .'8;03 - Swinming. 

(1)-. §04. - Basketbairr--- ^ 
,(!)• 805 - Modern Dance 
• (1) .806~:- .SfricanrDance 
: (1) 807-;- Fencing 

(1) 808'- - We-ight Training 
.. (1) 809 - Exercise 

- Cih-BiO - Tennis'-.^:' -/ - 
■ (1) 811 - Karate J 

(D- ^812 --. Fitnesjs •. ' ' 
' (1) ' 813 - Color Guard\ 
(1) 814 -■ CheerleadingN, " 

(1) , 815 Danc^ , 
d) 816 - Soccer.- .* ^ 

'8i«7 - Volleyball - 
.818 - Roller Derby 
IX) 819 - JuJitsu 
J2) 820 Yoga- - - 
H2) 821 - Ice Skating" ' - 

(2) 822 - Bowling . 
(2) 823^,- Elaine Powers Figure Salpn 

(2) . 824 \ Aerobics " 

- (2) 825 -^Kung;Fu 
(3> , 826 - General Activity 
0) 827.- Softball 

(3) 828 - -Cycling 
(3) . 82? - Golf \ . » . 

. ^4) 830 - Aikido \ 
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900 Level. - Mlsce:llaneoudv 
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' (1) .^01 • Independent Study - Sewing 
S^, (1) 902 - Independent, Study - Writing " 
'IP 903/- Independent Study - Photography 
; U) 904 indegendent Study - Environment 

<1V 905 Independent Study - Auto Mechanics 
\ iiy 906 - independents Study - Drums ' . 

./(2) 907 - Independent Study - Sociology of the City 
/C2) 908 - Independent Study - Macbeth 
/ /'/(2j^ 909 - Independent Study - Daijce^ ^. 
l/Y'r (2) 910 - Independent Study - Auto Mechanics II 
'^^^'"^f^y ^^^^ " Independent Study - Special 'Brqblems in Psychology 

/^^^ ^-^^ " Independent Study - Drti&s II 
\^/7 , (4)^13 - Independent Study - 'Business Math 
.V y , (4) 914, - Independent Study - .Creativity 
> r ^ (4) 915 - In'dependent Study' =^ 'Health 

(4) 916 - Independent Study/- '^r^.ting Ri|search' Papers 
(4) 917 - 'Independent Study)\-. History . 
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• • The Individualized Learning Ci^nter activities were conducted by the^^ 
Instructional Sysfeo^-it^f . The Care^e^Wl'oratioh and Specialization 
activities were coriduicUly ^artici^^rihg employers with the assistance of 
the fmployet Systems sta^^v/lhe AOO. m and 00 level elective courses were 
conducted by volunteers frotrf. Research fo^TJetter Schools, participating em- 
Ployers and other agencies and individuals f\om the community. The Physical 
Education activities were conducted mainly by [the staffs of area recreational 
facilities. , . All learning activities were cobrd^-ed and supervised by proj- 
ect , staff members, .-^^ . 

.(^^\- Most individual eiiployers represfeiited one of the three members of^a 
Career Exploration Cluster. Each cluster provided learning activities for a 
quarter, with each employer offering instruction for one ^ull' day ±n four 
consecutive weeks.. There were deviations •frqm ^H^^'I^teVr^-t ^ general ^ 
each employer was active for one mbnth^^tJch quk|t^r^, Empf^i^^ .also had ' 
direct involvemen^wip,-^rtts through Career S^e^t^^^^^^^^^^^^^ 
not offered by aiTemployersH^hich were not inltil^^dXlihe second ' 
. quartet of operatic^ns. The principal employer-staff interli^^s occurred 
^gygh :the ,FiM'd;:c^^irdin4 who oriented employers .^sr^^ln employer' " 
W^^J^^v,^l^^^a^nt^ and_ closely monitored ^program' operation\^^y^i general, 
sessions and cluster m^etiil&s^ere also heldl vX-^^'--- ^ 



No uniform or structured orientation or in-service program was^^^ie^;* 
to staff nie»t,ers. Their involvement in the student program was, of c^rs^; 



a full-tij»e'^:^opositi6n. The operations staff had primary responsibllit'ii^^for , 
the on-going Academy Activities, while the developmental staff was tobe-feS^" 
gaged in« lot|ger tange.concepWlization and research. In fact, this func^i^yi 
distinction =pould not be' maintained due to the extreme press of operational .'W 

ill ; I • 's-'l*' 

; t / .■ . • .-f^ 



need^and issues:. A staff retreat and occasional general staff meetings were 



'. hfeA to. 



,6, disseminate infonnation and promote interaction. Leaders within com- 
ponents also conducted smaller sessions on a regular ba^il to keep staff in- 
formed and working on a common basis. 

... For a more detailed discussfon of these treatment application activities 
see the milestone series in each area, the "Operating Plans" (July 2C , 1973), 
: oc p^ftihent Evaluation Reports (indexed in Appendix' .A) . 

, . Regarding the collection of information, the standardized tests and 
Student Questionnaire were administered in a. pretest-posttest design at the 
< beginning and e^d of the academic year by the evaluation staff. The Con- 
prebenslv^ Tests of Basic Skills were administered in special large-group 
sessions; \he Personal Orientation Inventory and Student Questionnaire were" 
administered ^n the Group ' Counseling Sessions; and the Test of Economic 
Unde/sta«iing'was administered in the World of Work Seminars!:- Results were 
Rapidly fed bitkrto appropriate project staff for studfent placement and . 
program developm'ent purposes." All data were then coded .and "^J^tlr^d into the 
Data Processing "System deyelop.ed over the year' for analysis purpose^s. The 
Student and Employer Information Sheet' data were collected upon entry into ■ 
the program, entered-for data p^pcessing and disseminated in evaluation i^jcrt 

The following instruments wLe indigenous to the present project, and 
were scheduled for monthly adml.nistr3tt.-on until developed to a point of dem-' 
onstrated stability, when'a quarterly administration cycle wovfld take effect: 
Student Evaluation Report, Employer Evaluation Report and Staff Evaluation 
Report. All reached a quarterly status by the second quarter, with some 
alterations of plan. The Employer Evaluation Report proved unworkable due ' 
to the facility with which written response requests may be postponed or 
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mislaid. This instrui^nt w^s\ccordingly reworked irtto the Employer- Eval- 
uation Interview Schedule for individual, personal quarterly .administration 
•by the evalAtion staff beginning second quarter. The written Student -Eval- 
. uation Report met with substantial non-response, and was abbreviated for 
oail administration second quarter to a randomly selected half of the student 
sample. .This instrument was complemented by the Student Evaluation Interview 
Schedule, which was individually administered to the remaining students by 
a contracted professional interviewer beginning second quarter. The Staff 
Evaluation Report functioned well as a written-response instrument and also 
assumed a quarterly cycle second quarter. The results of these measurement 
processes were disseminVted to staff and project subjects involved through 
evaluation reports. Data were not entered into the Data Processing System as 
their bulk would not have allowed time for coding and processing given the 
small size of the evaluation staffs " ' , . 

The Career Exploration Questionnaire was developed during the year but 
retained a monthly 'cycle since that coincided with the duration of individual 
employer program^.. ^ This instrument was administered by the evaluation staffi 
in the Group Counseiing Sessions. Results were reported t6 staff iri'eval— 
uation reports an"a to students in' special reports designed for discussion in 
the Group Counseling Sessions. These data were also treated manually. 

For a mote c9mplete discussion of the instrumentation process, see Mile- 
Stones 10, 11 and 12 "Evaluation of the Operations of the Academy for ^Career 
Educatton" (June 15, 1973). 

The results from all instruments were disseminated to all staff members 
in the form of the evaluation%ports indexed in Appendix A. While these 
findings were frequently discussed in the weekly Project Cabinet meetings and 
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with individual ^aff members, no structured follow-up mechanism was devised. 
It is therefore known that all data were availifife 'to project participants." 
but^it is not known with any comprehensiveness the manner and extent to 
which the data were used.' It was also not possible to tailor the rep.orts 
to different audiencesr or spend adequate time on deriving consequences and x 
recommendations for program development . These two. limitations diminished 
the utility of the first-year evaluation effort. 
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; RESULTS AND DISCUSSION 

Data. Analysis - . ^ . ^ * * 

Results are presented^in three sections. -The first \s a discussion of 

• standardized test results. These reflect on the project as a whole, and 
identify to Vhat extent progress has been made in traditiortally concijived 
academic areas. The sec oncj section addresses point-by-point the objectives 
from the Evaluation Plan stated above under "Program ^a Is". In this ''section 
the data available are more subjective and diffuse. ^ut each point is covered 
in as diligent a manner as possible. The last section discusses other 

• issues Which were not -formally included in the evaluation plan, but which 
-merit attention in this year-end re'port. 

The major standardized test utilized was the Comprehensive Jests ot 
\ Basic Skills. This instrument was designed to measure performance in the 
- cognitive areas traditionally construed as Reading. Arithmetic and Language. 
Sincejhe item- content has no intended coincidence with Academy learning 
sequence content. tKe instrument is purported to mefsure generalized knowl- 
edge. 



During the Orientation Program preceding the^ first year of Academy 
operations all attending students (n-107) were administereXhe Comprehensive 
Tests of Basic Skills (CTBS) by the evaluation staff. Due to\sences 
from individual sessions the data available 'for any one subtest Presented 
approximately 100 students. Near 'the end of the academic year, duAng 
two special'sessions in the middle of June.' all attending students wL / 
administered an alternate form of .the CTBS by the evaluation staff. Aese 
sessions were each attended by approximately 70 of 'the refining 90 enr\lled 
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""^.ucs. Matches avauatu W«ea. aW p„"a„eat .a.a rea„Iead 73 
^aca palra ,or «ea.i^ a„. A^Uh^elc a„. 57 .aea palra Unsuaga a„. 
Total Battery. „„,e Reading aad Arith.etlc ^acorea «ra avallaba^ for " 
•analyala because aUdeata who. „issad theae aubteata 1„ eha fl.at aaaaion ^ 
were p„.aued i„ ehe aeco„d aeaalon evea though thla detraoted trcn the 
co^leteneaa^of othe. aubteat ..data. 'Thla „aa do„e becauae of the pe.eeived' 
focal i„po„a„ce of Keadlag i„d A^lth^tlcachleve^ut. Thua the .eauUa ' 
oa Reading a,d A.Uh.elc galaa re^eaent 77. of .he Academy atudeuf body. ■ 
While language and Total Battery galna iepreaent only of the Academy 



^Students, 



,;. Table 1 preaenta a„«ry reaulta 1„ Grade Equivalent f„r„. Aa can 
be..aee„. atudenta- gained an all aubteat areaa, except Arlth^eflc AppUcatla^- 
with a .ocal .ea* ga»„- of 0.54 of a grade level. . galna In feadlng • 

vocabulary. Arlth^.lc Co„p„«rlo„ ,nd language Spelling .ere particularly 
high. Game m Arithmetic Appllcatlona .ere negative (atatlatlcally not 
-"ferent,. .Kile galna In Arlth^rlc Concepta.,nd' Unguage Mechanlca ' 



Were particularly low. 



^'"^a 2 preaenta au-,«ry- riaulta In Scale Score form. Theae galna ' 
were aubjected to T-atatlatlc analyaea aaau.l„g e<,ual but. unkn«.„ variance 
The laat column of ehe table preaenta t^e level.^of coi,f Idence „Uh „Klch • ■ 
tte mean galna may be accepted aa not ^.ue to chaVce variation. Although 
"O ac.ep.able level of confidence .aa pre-apeclfled. the .90. level la ■ 

considered reasonable for thp i„in^i u 

the initial phases of experimental project analysis. 

Using this criterion all total score gains wo^ld be considered- reliable ' 
The following additional .ub^^st score gains would meet this criterion: ' 
Reading Vocabulary, Arithmetid^ Computation, Language SpelUn^. • The . 



Table >1 

Pretest - Posttest Comparisons on the CTBS 
in Grade Equivalents 



Variabl e 

, ^ !• Readipg 

A. Vocabulkfy 

B. Comprehension 
C- Total 



m Arithlnetic 
' A. Computation 

p. Concepts 
' C* Applicatiohs 
Df Total 



III. Language 

A. Mechanics ' 

B. • Expression 

C. Spelling 
D- Total; 



Battery 



Pretest Score 



■9.04 
8.49 
8.78 



7.61 
8.07 
7.70 
7.69 



Posttest Score 



9.75 
8.99 
9.37 



8.04 
8. 12 
8.27 
8'. 00 



7.94 



8.29 
8.26 
7.63 
8.11 



8.37 
8.61 
9.18 
8.64 



8.48 



Change 



+0.71 
+0.50 
•=,+0.59 



+0.68 
+0.19 
-0.07 
+0.42 



+0.33 
+0.49 
+0.91" 
+0.64 



+0.54 



•IiiaU cases. for Readi-ng and Arithmetic n=73; for Language and Total Battery 
n=57._ This disparity occurs due.to the decision to admi«lster Reading ,nd 
Arith«etio;te3:t^''at the expense^of others due to their perceived 'greater ' 
significance for the program* 



< 



^ TABLE 2 

Pretefst - Pbst4:est Comparisons on the CTBS 
in Scale Scoresl 
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Variable ' 


Pretest Score 


PosttQst Score 


Change 


Confidence 
Level 2 


'S ^ 

I. Reading J^ 

A. Vocabulary ' 

B. Comprehension 

C. . Total 


568 
557- 
562 


597 
^ 570 
* . 582 


+29 
+13 
+20 


.97 
.80 
.90 


II. Arithmetic 

A. Computation 

B. Concepts 

C. Applications 

D. Total 


507 

524 
'504 


532 
533 
521 ^ 
525 - 


+25 
+ 9 
' - 3 - 
+21 


. -97 
.73 
.61 
.92 


III* Language 

A. Mechanics 
, B. Expression 
C. Spelling 
Total 


* 

542 

548- . • 
549 

537 . 

. ■ V 


553 
564 
576 
. 561 


+11 
+16 
+27 
+24 


.75 
.82 

<"96 , 
.93 


IV. Total Battery 
^ 


524 ,," 


545 

# • ■ 


+21 


.90 



fcoJL fni^lf ? f ""f "^^ °^ ^ distribution of 

Ltl^HcSn J °^ ^^^"^'^ «^he mean was 

established at 600 and the standard deviation at 100. Possible scores 

sSflstl^llJ^?''"'^'"^''*^^ ^° '^^^^^ •^^^ "^ff"l 

2. Confidence Level refers to staClstlcal probability, or the reliability, of 
findings. A finding with no possibility of occurrence by chance alone 
would have a confidence level of 1.00. These confidence levels were de- 
rived using T tests for pfedlcted differences in one xllrectlon 



^7 



35, 



following subtest scWefeains could not be considered reliable using this ^ 
criterion: Reading Comprehension, Arithmetic %onc^pts. Arithmetic 
Applications, Language Meclianics, and Language Expression, 

Although in some respects the data are clearer than anticipated, 
their interpretation still remains problematic. Assuming that available 
data were representative, it is clear '*that, in geneVal, students progressed 
in traditional academic skills during the course of the year. It is also 



apparent that some performance areas were 
In reviewing this difference, by area it 



ted better than others, 
^eeins. that ,replicatlve knowledge 



such as vocabulary and computation skills wa^-affected more positively 

than interpretive knowledge such* as comprehension and concepts skills. 

This is not what would be expected given the nature of t-he program; but' 

it may be due to student type, Reading and Language were also affected 

more favorably than Arithmetic. -This would be expected given the nature^ 

^^ft^?^ program. These observations have Internal program implications, 
I 
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The nextl^ltask was to Investigate the%e evidenced gains in terms of how 
they related to what 'gains toight have been expected, ajid how they related to 
possible predictor, variables. „ ' 

In t^e absence of coiiparison cgroup data and local normative information 
(which ma^become available at a later date) , 'nati9nal normative "data were 
used in aii attempt to provide a context for expected gains. Based on year- 
end scores Academy students were on. the low end of the percentile range 
for both Total Population and Large City Subpopulation Norms: 





Test 


Total Perceraiile 


Large City Percentile . 


1. . 


Reading Total « 


33 


35 


2. 


Language Total ^ 


?6 • 


28 


3. 


Arithmetic Total 


20' 


■ ; " 23 v 


4. 


Total Battery 


22 





The 25th percentile norming group was selected to establish gain score 
expectations. First, 11th «rade. gains in terms of scale scores were determined, 
These repreaent the gains actually evidenced 1>y the natidnal Tibtming sample • 
over the'cQurse.of a school yeat. These s.<;ores were then adjusted to account 
£or test-retest discrepancies on different test forms. The adjustment figures 
were derived from two-week- interval test-r?test data on form Q Levels 3 and 4 ' 
(data provided by California Test Bureau). The .results in scale scores Here 



(4 

follows : 








^ Test 


Expected 
Gain 


Expected 
Gain Adjusted 

V 27 


Actual 
Gain 


1. Ifeading Total 


38 


20 

24 ' 


2. Language Total 


7r 


2 


3. Arithmetic Total 


' 29 . 


15 


21 , 


* 4. Total Battery 


32 


' • 16 


21 



The Expected Gain Adjusted figures represent the scale sbore gains Evidenced ' 

by the. 25th percentile .subgroup of'£he national no'rming /ample adjusted to 

• ' ♦ ' • , * ' .• .it- 

jreflect* inter-fonn differences. -The Actual Gain figures, represreht the mean 

|cale score gains evidenced by Acad.emy stud^ts. As ,can be seen from these 

comparisons, the Academy students exceeded the expected ^owth rates ip every \ 

case but Reading Total. This would sugg^s^that the academic progress^ exr 

hibited by Academy students Vas greater than that' which woiild have ^occurred 

in public school. ' «^ . * ' ^ / • O / ' 

* ' *\ . ' * 

Achievement gain scores were also analyzed' to ^determine whether or nor" 

gains were dependent uppn student -chauacteristics^. Table 3' presents the', 

results, of correlations involving selected -student ,d&nogr^ph£i:. and entVy .• ' ^ 

level characteristics, ^ex group' membership showed- 1 '^significant Language ' 

subt€!st. relationship and 2 .significant total score 'relationships':' The- > 'T'^ 

jimplication is that females tended to improVe moVe than males' on some language 

- - " ' ' .. < ^ 

tisks. Age yieMed only 1 significant relationship indicating a'^tendency 

ft *' ' ^ - ' - ' • • 

for younger students' to gain mo^e than ol<|^studei>ts. '. Racial' group member- 

ship,. was found to/be totally unrelated^ to gains. i.Q. showed .significantly 

positive relationships with Reading Vocabulary and 'Arithmetic Computation. 

- ■ • • - ^ ~ ' 

but a negative relationship with Language St)eliirig. Vrete'st scores yUlded • 

a significantly positive coi;relation with Readiog ^ocabuliry but signif- ' » 

icantly negative ones witb .Reading Coiffpt'^hens ion , Languag^ Mechanics and " 

Language Spelling. " '^e lack of strong and consistent relationships foui^ip ' 

these analyses indicates that the Academy progr.am hds not hid general dif- \ 

ferential effects ,^on students as categorized by the variables selected. . ' 

Further analyses were preclu^ by this, finding. This, has very positive 

'. 1 / - . - • * • , 

implications for the Academy program in that soioe population giibgroups' are 



' Table 3 

* { 

Correlations Between Xchlevement Gains 
.and Selected Studen^ Characteristics 





Gain Scores — 


1 




1 

^- Race 


I q 


Pretest 




. I Reading' . * . 

A. Vocabulary • ' 

B. Comprehension 

C. Total 


1 

.03 / 
-.03 ' 
.00 , 


-.17 
-.12 
-.22 


-.04 ' 
-.01 
.00 


.29** 
-.01 
.18 


.-29** , 
-.33** 

.07 . . 




II. Arithmetic 


/ 












A. Computation 

B. doncepts 

" C.'^^^^>lications 

B^^jC/Total 


.02 
-.15 
-.09- 
-.04 


-.16 -.10 
-.27** -.11 u- 
-.02. . -.13 
-.20 -.07 


.26* 
.19 
-.10 
.22 


-.11 
-.04 
.15 
-.11 




III. Language^ 










^: ♦ 


. A. Mechanics 

B. Expression ^ 

C. Spelling 

D. Total 


* -.27* 
-.02 

-.isli 

-.29* 


.25. 
.00 
.25 
.24 


.18 
.00 

.13 


.05 
.16 
.44** 
•02 ^ 


^.01 
-.61** 
-.22 




IV. Total Battery 


-.32* 


-.12 


.04 . 


.17 


.16 




* p.<.05 where R(X,Y) >.23 (df 


-70) 


and .26 (df 


m 


T f 

^ 




** p.<.01 where R(X,Y)> .27 (df 


- 70) 


* 

and .34 (df 


» 55) 





In cases where variable Scores wetie categorical rather than scaled 
(sex and race) artif iclml d^ahotemous variables (X or not X) wtre ' 
established for analytic gjjrpofiles. Correlation figures are reported 
for one category only, indicating a p6sitlve or negative favoring of 
fhat category. , * 



apparently not favorfed over others with ^regard to what they^may be ex^ifiAted "^ J"../ / 
to gain from the program. Further the negative relational >t>deniy eVld,encted '/^ V ' 
by pretest scores -would Indicate that," at least In several areas, the- students / 
who need It the most are gaining more. ' " ' . 

Another variable that was posited as having an effect on academic progress 
was student attendance. Tables 4, 5, and 6 present the cor relations -of at- C 
tendance data by course and quarter with achievement gain scores.* is 
apparent from these analyses that course attendance bore lio consistent re- 
lationship with standardized test gains. The persistent lack of significant ,' 
relatlonshlp ^P^t hls area 4s striking, but not enigmatic.^ As <iscussed in ^ 

numerous evaluation reports the attendant data itself cannot be considered / 

* • ■ 

reliable, and chance variation here could obscure relationships. It is also/ // 

, I I ■ • 

possible that attendance variations only in extfepies wouldhave facilitating ' 

/ ' . ' ^' 

or debilitating effects on the J)rogram's capability ^to produce gains/ The / /T 
various hypotheses suggested by^these data will be the topics of analyses ^ ^ 
during FY 1974. - • ' • "^/Z / ' . 

In summary, it can be stated that Academy students gained In most Vi-a-/ 

»• " ./ ' ' ^ 

dltlonal acadeinic skills through a n'on-tradltJLoiLal curricular experieace./ 

Ac^Ufevement gains measured by the Comprehens^-ve Tests of Basic Skills seemed 
to be both- statistically- .reliable" and ^iractically significant . The availabiliLty • 
of compi^rison |roup data and normative group data would perml^ the drawing of. 
conclusions with more directness and clarity. 



Table 4 
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Coi|relatioiis Between Achievement Gains 



•;• v.; and First Quarter Course Attendance 

i 



pur ses 
_igaial5AQres-. 



Tv- -Reading 

~ .jAi.— Vocabulary 

B7 ■ ■'•Ceopiehen^cm' 
:J» ~C;. - To;taJL_.. .- .j^- 



ILC- 
^gligh 



.01-' 
.05 • 



iLe- 

Math 



'<Q1:' 

•- ■.0 8 - 

;-:*oV;. 



¥OWS 



' CE . 
''1 



.'1*0 ^-^.-zo* 
.09:::->^.09 
•oi; i:2o 



.10 

-.08 
.01 



CE 
3 



.04 
.09 
.07 



WIL 



.14 
-.04 
.07 



XI. AritlWietic • f 

, A. ^ C5>f^ujCation 

B. .flqiicjs^s . 

C. Ap|pljLe:a;tion^ 

D. Tob^Fr;.' . 



III. Language- v.-- 

A. Mechanics-: 

B. Expression . \ 

C. Spelling 

D. Total • 




.20 
.23 



.11 
.02 
.20 
-/.22 




.06 
.11 
.08 



.03 



*. P.<.05 where R(X,Y)>.23 (df - 70) and .26 (df »'< 55) 
** R.<.01 where. R(X,Y)>. 27 (df-70) and .34 (df = 55) 



.05 


-.02 


.09, 


. .26* 


-.07 


-.11 


.18 


.16 


.15 


.05 


.12 


.12 


.11 


^ .09 • 


.20. 


f32* !: 


^1^ 


.16 


.28* 


.29*r'./: 



ILC - Individualized Learning Center 
WOWS - World of Work Seminara' 
CE - Career Exploration rv''. 
WIL - Workshops in Living 
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Table 5 

Correlations Between Achievement' Gains 
and Second Quarter Attendance' 



bour ses 
Gain Scores 



I. Reading 

A. Vocabulary 

B. Comprehension 

C. Total 



ILC 


ILC 




EnKlish 


Math 


WOWS ' 


* 

.15 


.14 


V 

-.17 


-.15 


-.13 


-.10 


^.02 


-.02 


.01 



CE 
1 



CE- 
2 



CE 



'A 

•VJ7IL 




— ..:v03 

Ol 

02 'Xi 



-05 
-.07 
-..02 



ill 

■;:^*:g5 

;;.o4 



II. Arithmetic 

A. Computation 

B. Concepts 

C* Applications 
D. Tbtai* 



.17 


.17 


.02 


.09 


.13 


.0? . 


.16 


.01 ■ 


.01 


-.09 


-.02 


-.02 


.06 


.03 


.07 


-.09 


-.06 ' 


- -.03 


-.07 


-.02 


.02 


.15 


♦ 14 


-.03 


.04 


.09 


.13 ' 


.12 




W/>^^ .* p.< .OSjwhere R(X,Y),>.23 (df = 70) and 

p»<»01 ^tHiexe R(X,Y)>..27 (df « 70) and 

. ' - ' <* 

ILC Individualized Learning Center 
V, WOWS - Worlrfpf-^'woric* Seminars 
CE V- Career :Exi^lorat ion 
WIL V Workshops/ In Living * 



.26 (df 
.34 (df 



55) 
55) 



v.. 
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Table 6 • 

Correlations Between Achievement Gains 
and Third Quai;ter Attendance 



1 HflW^"'" 

Galnt Scori f ' 

fceadlngt . 


ILC ILC . CE CE CE 
; English Math WOWS 1 2 3 WIL 


A. Vocabulary 

B. Comprehension 

C. Total _ ' t 


-18 -,17 .20 .08 .13 .0^ 
.11 .09 -.03 -.01 -.01 -.08 
•21 .20 .15 .13 .04 .09 -.07 



A. Computation 

B. Concepts 

C. Applications 

D. Total 



A I Mechanics 

B, Expression 

C. Spelling 
D- Total 



.04 


.06 


.09 


.10 


.08 


.03 


.04 


.05 


.20 


.02 


-.19 . 


-.19 


-.09 


.12 


-.04 


-.03 


-.01 . 


'.09 


. .20 


.04 



•12 
.18 
.03 
.18 



.27* 
.05 
-.03 
.20 



.28* 
.03 
.06 
.19 



.06 
.03 
.02 
.02 



.06 
.06 
.07 
.08 



.05 
.06 
.18 
.13 



-.18 
-.14 
.15 
-.16 



03 -.03 

07 >*.06 

05 .20 

03 .03 




p.<..05 where R('X,Y) > .23.=^df = 70) and .26 (df 
p.< .01 where R(X?V^>.27 . 70) and .34 (df 



55) 
55) 



ILC - Individualized Learning Centfer. 
WOWS - World of Work Seminars , . > 
CE - Career Exploration 
WIL - Wo|Jcshop.s In Living 
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The two remaining standardized -Instruments were selected to- measure ' 
economic conceptual knowledge the focus %f the World of Work Seminars, and 
> non-cognitive dimensions, the focus of the Life Skills Explorations. 
These instruments were the Test of Economic Understanding atH the Personal ' 
Orientation Inventory. 

The Test of Economic Understanding was designed to fill a measurement 
gap "in assessing student understanding of the basic economic concepts es- 
sential for good citizenship." Alternate forms of the test were adminis- 
tered to Academy students in beginning and end of the year sessions of the 
World of Work^^mlnars. Summary data are presented in Table 7. Students' 
exhibited sm^ll tut consistent gains which were statistically significant 
(p.<.05). The actual size of the gains would seem to make any practical 
significance unlikely. The test instrument itself was found to have several 
limitations." The reading level required to understand the questions, re- 
gardless of knowledge of the concepts involved, seemed to be too high to 
allow general applicability of* the instrument to urban high school populations 
Correct answers were constructed based \ipon particular theoretical orien- 
tations which are neither universal nor • constant; the item validity is thu8>.^ 
^4endent upon "point of view" to a greater extent than wpuld be desireable. 
The normative data provided also raise some questions regarding applicability. 
Although no time period was indicated for the collection of normative in- 
formation, since the test was published in 1964, it is likely that the norms 
are 10 or more years old. The normlng sample was also biased against urban 
students. Only 25Z of the schools included were located in cities, and the 
definition used for city wa^ a population center of 15,000 or more persons. 
From this information it cannot be known' whether any large cities were -in-"* 
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'Table 7 
Test of Economic Understanding 

Pretest Posttest 



Raw Score Mean * 

Standard Score Mean 

Percentile Score Mean 
t 



\ 



16.25 
11.31 
'15.99 



17.82 
12.35 
18.72 



Change 

+1.57. * 
+1.04 * 
+2.73 



I 



co?;eiatef sIlL^^-'' " ^! ^^^-^^i^tical test employed wis a T test for 
Wis for on^J^r^r! "'^"^^ but .unknown variances. , Confidence 

i.eveis tor one-tail distributions were used. 



\ 



i 





V eluded in normalization. These considerations -render the' obtained results 
,of limite<^ usefulness. ' 

The Per^nal Oi^ieiit^tdQiLlsven^^ affectiv^ instrument, based 

upon Maslow's theory of the self-a^^^^^^^^i;:^:^;;^^ this th£r\the self- 
actualizing individual is seen as "developing and utilizing all of his unique 
capabilities, or potentialities, free of the inhibitions. apd emotional tur- 
moil of those less self-actualized." The test is composed of. 14 sibscales as 
displayed in Figure 1. Academy students were given the Personal Orientation 
Inventory during beginning and end of the year Workshops in Living/Group 
Counseling Sessions. The results are presented in Table 8. From these data 
• it is apparent that students gene rall^^^ became more -"self -actualized" over the 
course of the year with 8 of 14 gain seores statistically significant 
beyond the .90 Confidence Letlft. Figure 2 present* the Academy student score 
profiles compared with scores deri>(ed from a high school norming group (sample 
characteristics unspecified). Academy students' scores compare favorably 
with these normative data. It seems, reasonable to conclude that Academy 
students progressed along the affective dimensions measured by the Personal 
Orientation Inventory during the course of the year, and that the resultant 
general profile demonstrated no sigi^lficant problem areas. 
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' . Figure 1 
Personal Orientation Inventory Scales 
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Number Scale 

of Items Number Symbol 

f L Ratio Scor.es 



Description 



23 



^2^ 



1/2 



3/4 



32 



23 



18 8 



16 ,C 9 



V^C TIME RATIO 

Time Inconjpetencc/ 
^ Time Competence - 

measures degree to which 
one is "pi't-^sent" oriented 

O/I SUP^^ORT RATIO 

Other/Inner - measures 
whether reactivity ori- 
entation is basically to- 
. ward others or self 



n. Sub-Scales 

26 5^' • k SAV 



Number Scale 

of Items Number Symbol 



26 



SELF-ACTUALIZING 
VALUE 

^lcnsures affirmation 
ofa primary value of. 
self-actualizing people 



Ex EXISTENTULITY 
Measures ability to 
situationally or existen- 
tially react without rigid 
adherence to principles 

Fr FEELING REACTIVITY 
Measures sensitivity of 
responsiveness to one's 
, own needs and feejings 

S SPONTANEITY^ 

Measures freedom to 
react spontaneously or 
to be oneself , 

Sr SELF REGARD 

^•Measures affirmation of 
self because, of worth or 
strength . ' 
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SELF ACCEPTANCE 

Measures affirmation or 
acceptance of self in 
spite of weaknesses or 
deficiencies ^ 



NATURE OF MAN 

Measures degree of the 
constructive view of the 
nature of man, mascu- 
linity, femininity 



SYNERGY 

Measures ability to be 
synergistic, to trans- 
cend dichotomies 

ACCEPTANCE OF 
AGGRESSION 

Measures ability to ac- 
cept One's natural ag- 
gressiveness as opposed 
todefensivencss, denial, 
and repression of 
aggression, ^ 



CAPACITY FOU IN- 
TIMATE CONTACT 

Measures ability to de- 
velop d^bntactful intimate 
relationships with other 
human beings, unxjn- 
cunibered by cxpectn * 
tions and obligations 
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"Personal Orientation Inventory 

\ . 



Variable 



1., Time Ratio < 

2. Support Ratio, 

3. Time Competence- . 

4. Inner Direction 

5. Self-A%:tual'izlng 

6. 'Existentiality' 

7. Feeling Reactivity 

8. Spontaneity 

9. .Self-Regard 

10. Self-Acceptance ' 

11. ' Nature of Man 

12. 'S}inergy 

13. Acceptance of Aggression 
143;, "Capacity of Contact-f 

« 9 ' .. '■>' 



Pretest 



21.43 
16.92 
14.28 
75.85 
16.97 
17.95 
14.34 
10.90 
11.48 
14.31 
9.64 
, 5. '54 
14.48 
16.23 



Posttest 



22.82 
19.73 
14 .^82 
79.91 
18.07' 
19.134 
15.27 • 
11.42 
11.47 
15.02 
9.49 
6.11 
15.40 
17.51 



Change 



+1.39 
+2.81 
+0.54 
+4.06 
+1.10 
+1.89 
+0.93 
+0.52 
-0.01 
+0.71 
-0.15 
+0.57 
+C.92 
+1.28 




Confidence Levo] 




'he statistical test employed was a T test ,for correlated sample scores 
assuming equal but unknown variances . , Confidence level r«^rs to the 
statistical probability piTreliability of findings. A finding with no 
possibility of occurrence by chance alone would have a confidence level 
of 1.00; __The 'confidence levels were computed for predit:ted differences 
in .one direction. 
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Figure 2 

Pfirson.l1 Orientation Inventory Profile 



SELF. 

ACTUALIZING 
VALUE 

Holds values 
of self- 
actualizing 
people 



EXISTENT! 
ALITY 
flexible in 
applicatiirn 
of values 



'FEELING 



FEELING 
REACTIVITY 
Sensitive to 
own needs 
and feelings 



SPONTA 
NEITY Freely 
expresses 
feelings 
behavtorally 



SELF PERCEPTION 



SELF REGARD 
Has high 
self worth 



SELF^ 

ACCEPTANCE 
Accepting of 
self in 
spite of 
weaknesses 



SYNERGISTIC AWARENESS 



NATURE OF 
MAN. CON 
STRUCTIVE 
Sees man as 
essentially 
good 



SYNERGY 
Sees oppo 
sites of life 
as meaning 
fully related 



INTERPERSONAL SENSITIVITY 



ACCEPTANCE (CAPACITY 

OF IfoR' 

AGGRESSION INTIMATE 

Accepts I CONTACT 

feelings cf 'Has *dfm 

anger or [interpersonal 

aggression [relationships 
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Tke discussion now turns to program objectives indigenous to specific' 

pr6ject components. These were presented in the Evaluation Model aS repre- 

'j, ■ « . 

senting basic entities, whose abserice or functional failure would adversely 

affect the Academy progr&n. ^atalrelating to these objectives are drawi 

largely from project-specific fonn|tive instruments and reports. • i, 

1*0 Employer Systems Component 

4 , 

1.1 Objective - Employers will extend cooperation suffidient for the provi- 
sion of career exploration and specialization. 

4 ^ - 

Results - A total of 46 employers were recruited for participation 
i|^ the program. They are listed by name (excluding the Chamber of 
Commerce and the Veterans Administration) above in the "Design" section 
and d^escribed above in the " Subjects" section^ With 10 to 12 employers 
offering exploration learning activities at any one time, this provided 
employer-based resources for students on ^Rasls of 1 employer for 
■ every 10 or si> students . , The variety of omployiirs involved also pewnitted 
the maintenance of the cluster structure. Specializations were also . 
developed, with a total of 10 in operation third quarter; for a listing 
see the "Design" section. The employer resources available were thus 
sufficient for the conduct of this component. 



1-2 Objective . r Employers will develop in the quality and quantity of their 
commitment. ? ' ' * 

^^^"^'^^ - Although this is a difficujLt dimension to measure and there- 
was consider|ble variation among ei#loyers, several generalizations may . 
be stated. From the beginning to -the end of the year the.- number of 
employers who e"Stpressed ^tlsf action with their involvement in. the ' ' 



• - li ^ 50. 

program Increased from approximately 40% to 70%. More than half of 
.the employers also actirely engaged in program revisions aimed at 
' improvement. Only 1 employer dropped" out,,during the course of the 

» 

^ . . additional 13 (28%) declined to participate in the succeeding 

year. Of the 14 total, 6 were not sought for continuation because 
. their input was not' considered ^equate, 3 hrd^inte^nal reorganization 
^ • problems, 2 felt that 'they had completed their commitment, ^ felt 
that the students lacked interest and motivation, and 2 needed ad- 
ditional resources tp continue. Thus, more than two-thirds- pf the 
employers involved demonstrated a continuing interest and commitment 
regarding the Academy program. 

• 1.3 Objective. .- Students will learn through their involvement in the 



program 
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~ it. is not possible to attribute leari^ing differentially 

to program areas, the gains discussed under standardized instruments 
apply to all program area's" including Eniployer Systems. Student qtjes- / 
tionnaires and -inCerviewis also indicated that approximately 80% of' 
the students were learning' about new^careers. These students, were" 
able to name specific jobs. they learned about; most :were able to 

! 

, identify the education and training 'needed; and about hklf also stated 
the afalaries associated with positions. One-third of the students 
|-TS.v5;dncountered a career that was ^f particular interest to them. ^ 

,1.4 Objective - A World of .Work Seminar can be developed -and conducted 

^ with the objecti^^ of p^viding^an overall view of exploration,^ infor- 
. mation, skills, concerns .and iss^jes related to career pursuits, *a8 well 

/ 

'ma. k ■ - ■ L , . ■ 



as a forum for student discussion of these 'matters\ * • 

J ' - ' ' ■ • ' i3t 

Results - These Seminars-Were developed and conducted by the Camber 

J of Commerc^e as a required course spahnin& the first three' quarter^/ : 
Much effoift, was devoted to constructing and revising this^activity,' " ^ 
but it was^j^rked by a poor redeption on the part of students. Tlie / 

, Seminars consistently had one <yf the lowest attendance rates for " 
Academy offerings. Only about half of the studeats saw^ them as I'earn- 
ing experiences, and, only one-fourth expressed 'a deSlce to attend them!^ 
When asked during the third quarter whether or oiot the-Seminars Kad 
improved Qver the course- of the ye'^r, only 25% agreg^f" Given these,--, 
relatively ne^atl^e a^ttitudes it is unlikely that the inteneiohs. 

t-^ • ^ ^ . V . ■ • , ■ ■ . ' 

the World o'fj^ork Seminars .^e .accomplished . \ \ ' 

InstTuctlonal^ystems Component " " . - ^ ^ - ' ' 

Objective - Instructional programs can -be 'developed and administered 

• • ^ ■ •i." •• 

to meet the students' academic needs as defined by State requ^'rementf , 
Academy- requirements and student interests. • 

Results - The array of instructiona^l activities offered 'during tfte 

firsfyear wa^pre^sented*above 'in -the *'DesignV section. As^an be ^ 

> * ' ^ . . , . . \ ' 

seeji^, these activities were numerous and diverse in "eacli area. The 

only required airea where pirpblems w^re encountered .was Physical Edti- / 

' ' " is ' ^ V ' ^ ^ \ 

cation. Despite the more than* adequate variety of *a9tivitie& to^^/elect! 
from, almost li^f Qf the 'students fai-Ied .to meet the 2 hours' per week"^ 
attendance requirement. In all are^s related to acad^nic credit. ^ ^ 

I y . ' , ' ^ • ^ • , " ^ . ^ ^ 

evident needs were met by .*the activities ^available:, 'On the N^udent ^ ^ 

'' ' " ' ^ " " * * * * 

Questionnaiy«'|bsttest of the' students^- indicated th^t their learning 



activities had been interesting; 95^, -considered th'ea relevant :/ &d 86% 
. ^ thought that they had learned' a lot as a result of^teajL^cad^y ex^' 

perience. ' • \ ^ 

.2 Objective - Psychomotor programs can be developed and conductfe^to - 
meet students' physical needs. 

Results - As ientioned above a superior Physical Education program was ' 
developed, but student participation could not be effected for a l^Stge 
segment qf the Academy student body. An assessment of psychomotor 
/ needs was planned, and most of ♦the pretesting was completed by the' 
Physical Education staff for placement purposes, but the absence of 
posttesting precluded 'any assessment of progress, 

3 Objective - A Life Skills Program can be devised and conducted to 
promote the development of skills in interpersonal relationships, 
critical personal values and other dimensions to he specified by staff 
and students. . . ^ 

Results - The Life Skills Program was implemented* in .the form of 
► Workshops in Living and Life Skills Specializations. The Workshops 
in Living were required small group sessions for all students and re- 
presented' the exploration phase of this area. The Life Skills Special- 
izations were elective offerings usually coriducted on an individual- 
basis with' the student contracting for certain activities with the -J ■ 
participating service agency . The Life Skills Program was initially 
a part of Instructional Systems, but du'ring the year it was transferred/ 
to Counseling Systems.. This was prompted by the .increasing emphasis on 
counseling as-an instructional activity and the replacement of the ; 
Workshops in Living by Group Counseling Sessions. Jn the process'- 'l " . 



ChlW fexplor^jtion activity focused less ok personality dovcvlopniont .md ' 

■ f.' . , * . ^ 

• more on conveying information and promotXnR skills', rc-latod to, I)otli 
career development and Academy operations'. The~l!v^erkshops in Living 
weife consistently rated very highly by students' while the Group 
Counseling Sessions were regarded as mediocre. /The Group 'counseling . 
activities were accordJ^a reception by stCidents that was consistently;- 
inf^for to Workshops in Living: fewer indicated that "I like par- 
ticipating" (73% vs. 100%); fewer thought that '-'They (Group Counseling^ - 
, Sessions) help me to think kbout myself" ^50% vs. 96%); fewer thought 

•that "They help me to understand other people". (AOX vs. 85%); fewer " 

agreed that "They're worthwhile" (50% vs. 85%) ;„ It is clear that i^t^ 
the change in organization a change in student attitude occurred, 
activity-specific measurement « of knowledge was included. T^^" 

Student Personnel Systems Component - ' 

Objective - A ^tudent records system can be deve£*oped -and maintained 
with the ckpabVlity of responding to staff' md evikaHon needs. 
Results - j;h\records system wKlch evolveycySlJ^tisfacfc^ry neither 
for staff nor evaluation use. A\comprehynsive\dyL:em w\^^^'esigned 
before operationalization, but it was never deveViJi^d "arfl^'^^^iek. ^] 
The consequence was that records and documentatior<*^^curf^ia .a > 
haphazard way, with components and individuals em'pl^-Slg esc^l^'c 
procedures. Iq some cases records within area's were sufficient for. 
operations in that area; across areas there was no coordination-. Two 
project functions experienced especially .negative effects as a result 
of this: Documentation in concrete and accurate terms became difficult 



. ( 



5^. 



t- ' 



V 
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•b^caus4. complete and reliable information, was seldom ^^;JailaMe\ Eval- 
;.uation was also impeded because extensive 'data reconk^ruction and 
^v^rlficatiopjhad to be completed before any evaluatio^^ld be ^tempted. 

3. ^.Objective - Student Counselors (Developmental Advisors) will V^viil- 
> to. advise students on career, academic and personal matters t 



, CHe students' satisfaction as they progress through the Academy. 
• ..'feilts - A great deal of . resource was devotejd to this objective, 

*•**...».;• / 

• 'Vwieh 1. counselor for every 30 or so students, and the activity was 
successful. Throughout the year the^/co^nseling program was rated by 
.students as one of "the best aspects/of the Academy. In response to 
. specific questions, approximately 1.2% of the students indic^t^ that 
, ■ "They, helped me solve problems"; ksX said that" "1 Hkd as njuch time as 
^ I needed"; 75% thought "They -helped me make dareer or e^lucational 
dedisions"; and;' 90% indicated get along weU%h my Developmental 
Advisor". v '. 
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^.3 Objective - Theystudent Personnel System will be able to adequately 

^^ place students in careers,. it secondary education or another secondary" 
school upon./tHeir separation firdm the Academy. 

-'"Altfiough no situd'ents graduated from the Academy, since all 
were juniors. 30 student^' wetfe^her dismissed or decided to leave.' 
A foliow-up.Uiaison with/ the ^blic schools was established for each 
^ student 'who- planned to /Return to another secondary schocfy It was, 
however ,::;o^^t*en difficult to* trace exactly what" happened to students 



. I, 



vdue ^hM±v elusivenefss'and the state of affairs of public school 
•.recojid^l ;:in any case^^na significant reentry problems were detected'^^or 

' • t • ♦ * " ' 
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4.0 Management Systems Component ' - 

4.1 Objective • The Academy faciXities and resources will be 'adequate for 



staff and student purposes-. ^ 

Results - Although Academy resources were indicated as adequate without 
^notable exception, the Academy facilities were the oj) ject of continual 

complaint .. The facilities were rated the poorest aspect of the Academy 

/' 

by both students and s^taff. Although some of the objections were 
spurious,* many were legitimate, underlining the difficulty of obtaining 

qualified private facilities for educational uses. 

/ > ^ 

4.2 Objective - The Academy administrative systems will adequately serve 
staff and student needs. ^ • * . 

Results - This referred to such functions as grading, credits attendance 
and scheduling. Each of these- administrative activities was more time- 
consuming and diffioult than Was anticipated. In' some cases the lack 
« • • \ 

of a coordinated records and documentation system added further com-' ^Jf. 



plications. Sometimes the systems employed were particularly prob- 
l^^tic for evaluation staff. Examples o^ this were tlTe assignment of 
^ different configurations of credit hours for the: s§me course, lack of 
explicit rationales and' delays in ftnformation; preparation. In general 
the syst.ems functioned adequately for students^,^ f ew complaints or con- 
> flicts having been observed. 

4.3 Objective - Students will develop in positi^^e attitude toward learning, 

positive self concept and positive attitude toward Academy experiences. ^ 
Results - As indicated above under standardized tests, the self concept 
pretest measure indicated tlo need for ipiptovemei^t since 'the Academy 
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Zudent scores were similar to normative figures. Results from the 
udent Questjtbnnaire directly reflected student attitudes. Although 
ly 31% of the students thought the Acadeny was "as good as (they) 

9 

f had hoped it Vould be", 83% saw it as better than their previous school. 
•^A total of htt liked eattending the Academy; 98% indicated that their 
learning activities were interesting; 86% thought that they had learned 
a lot in the Academy' progran. Attitudes toward specific learning 
activities varied widely from activity to activity. Tnose noteworthy 
due tp exceptionally high or low student regard have been 'discussed 
under the appropriate component ^reas^. 

^ Each of the objectives discussed above has been covered only in a 
summary manner. The extent of formative data gathered permitted several 
approaches to most of the issues raised. The approach presented was selected 
^ on the basis of perceived representativeness of,.l^e data, illustrative 
value and 'comprehensibility . 

y 

The last section of "Data Analysis" deals with issues which were not 
explicit in the Evaluation Model, but which became a special interest during 
^ the year. The firfet of these is student attrition: 

-Between the opening of the Academy and^ August 6, 1973, thirty (30) 
students had left the^Career Education Program either by dismissal ^r 
voluntary withdrawal. ' This coijstituted a 28% rate of attrition covering 
the school year and approximately half of the suniaer. Of these thirty (30) 
students, thirteen (13) .voluntarily withdrew with the following intentions: 

1. 8 students planned to return to their old school or another 
secondary school 

9- 
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2. 3 students intended to quit school altogether 

3. 1 student was getting married " ' 
4^ 1 student moved out of the area and presumably would 

resume schooling in a new location 

Seventeen (17) students were dismissed from rhe Academy; their plans were 
as follows: 

1. 3 students intended to return to another s^^dary school 

2. 14 students' intentions were not known at the. time of 
last interview 

These students were dismissed for the following reasons: 

1. 11 due to exceptionally poor attendance and low motivation 
^ regarding the Academy program 

2. 4 due to behavior problems and associated poor attendance 

3. 2'-du^ to insufficient credits and refusal to attend the 
fourth quarter- . . 

.Exit interview forms were available fpr only ten (10) of the students 



who had left the Academy. Although ^hese accounted for only 30% of the 
total, counselors felt they were representative, and a summary Review 
seemed warranted. Of these ten (10) departing students, only three (3) 
left the Academy in favor of another school. Of" these only one (1) was 
expressly dissatisfied with the Xcad^ajr, while the others felt that they 
coul^ better fulfill their needs at the:S ofd schools. The remaining 
seven (7jl students ieftj|the Academy for reasons not directly related to 
its program, i.e. they probably would have dropped out of any school, and 
perhaps the Academy postponed-.that action for one year. * 

\ 
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• .1 

Students left the Academy as follows' during the quar^ sequence: 

. / - • • 

1. First Quarter - 4 students 

2. Second Quarter - lo students 

* 3. Third Quarter - 4 students ' ' 

4. Fourtl} Quarter - '_12 students 
I • 30 

The characteristics of the students who were separated from the 

Academy and those who remained enrolled were compared." Tables 9 and 10 

present 'these data . There was no observed difference between the two - - 

groups with regard to sex and type of school previously attended,. With* 

reference to ethnic group membership, the separated gjroup was 50% Biackv 

while the remaining group was 66% Black. This caused a shift from 61% 

to 66% Black for the Academy student body. 

■ mm - 1 

Examining the separated group's past school performance, it is 
evident that they '{iid not do as well as the remaining group in their " 
sending schools, The separated group had an average attendance rate of 
84% and an average GPA of 68.- compared with figures of 89% and 75 for the 
remaining group. 

In terms of measured intelligence and basic skills achievement 
CAcademy administered), the separated students had soiewhat lower scores 
than the ^maining students. The_average IQ score for the separated 
group was 92,2, while the same statistic for the remaJLning group was 98.0. 
Likewise, the Comprehensive-Tests of Basic Skills scores 4or the separated 
group were uniformly lower than those of the .remaining group. Specific 
mean scores for the CTBS pretest in ^rade" equivalents are presented below: 

f \ - ^ 



ERIC 



59. 



V 



Table 9 



Students Separating From The Academy 



of Students 

i 

1 ' 

2 f 
3 

4 

"l . 

7 . 

8 . 
9 

10 

11 

12 

13 

14 
. 15 

16 
•17 

18 

19, . 

20 ■ 

21 

22 

23 

24 
. 25 

26 

27 

28 

29 

30. , 



Student // 

1006 
1028 
1002 
1049 
1041 
1018 
1011 
1105 
■ 1083 
1106 
1045 
1099 

1030 ^ 

1103 

1031 

1038 

1107 

1019 

1005 

1013 

10^5 

1.032 

1048 

1073 

1074 

1085 

1089 

1090 

1086 

1012 



Date Withdrawa 

« 

11/22/72" 

12/1/72 

12/5/72 

1/1/73 

1/16/73 

1/19/73 

2/2/73 
■ 2/12/73 

2/15/73 

2/21/7.3 

3/19/73 
.3/29/73 

3/29/73 

3/29/73 

4/13/73 

4/13/73 

4/13/73 . 

4/13/73 

7/9/73 

7/9/73 

7/9/73 

7/9/73 

7/9/73 

7/9/73 

7/9/73 

7/9/73 

7/9/73 

7/9/73 

7/20/73 

7/25/73 
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Table XO 



Compared Charafcteristics of Separated and 
Remaining Academy Student Groups 



Variable 



Quarter 



Separated Group 

JL ^ 



Remaining Group 



% 



A.l 


4'- 


4 


104 96 


B.2 


14 


13 


94 87 


C.3 • 


18 


-17 


90 83 


D.4 


30 


• 28 


78 72 


> Sex 


IS* 

-* 




* 


* 


A. Male 


/> 
15 


-..JO . 


39 


50 


B. Female 

% 


• 15 


50 , 


39 


50 


\ 






78 


100 


\ 

Ethnic Group 


• • 

• 








A. Black ^ 


. 15 


. 50 


51 


66 


B. White 

C . ' Other 


Z_i 


47 


27 

_C 


• 34 

10 




30 


100 


78 


1 100 




->>>. 








Sending School 










A. Pub 116 


22 


73 


56 


1 72 


B. .Parochial 


5 


17 


. ^ 18 


23 


C. Private 


- 1 


3 


> 

1 


1 


D. Not, Aval lab le\^ 


_2 


._7- 


-1 


A- 




30 , 


"100 ■ 


1 •• • 78 


. io6 


♦ * 

♦ 

Academy Ql Grades 


» 
















$ 

A. High 


* • 
- -2 


7 






B. Mid 


13 


43 







C. Low. 

D. Dropped 



11 
4 



37 
13 



Separated Group 



/ 

/ 



Variable 
VI. Academy Q2 Grades 

A. High 

B. Mid' 

C. LoiJ 

D. Dropped 



VII. Academy Q3' Grades 

A. ' High 

B. Mid 

C. Low 

D. Dropped 





_%_ 


^ 2 


7 


6 


20 


9 


30 


13 


43 


30 


100 


1 • 


^ 3 


5 


17 


6 


20 


18 


§0 


30 


100 



Remaining Group 
// % 
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Remaining 
Students 


Separated 
Students 


Mean 
Dif^^ererice 


1. 

i 


Reading d i 


8.7 


- ■ .4/ 


I. 


Arithmetic "8.0 


7.2 


- .8 


' '3. 


> 

Language * . , 8.4 


7.3 


- -l-.l 


4. 


Total Battery 8.2 


7.5 ' ^ 


- .7 


An 


examination of the grades achieved* at the 


Academy revealed that 


the s^atated students performed less veil than the remaining 


students. 



This should be exp^t^d^since seventeen (17) of the thirty (30) were 
dismissed. When obtained grades wer.e divided/into High (A or B) , Mid (C) , 
and^ Low (D or F) groups, it became, apparent that few separated students 
had high grades during any quarter. Remaining studerits*attained overall 
average grades in .the C+ range across quarters while separated students 
earned overall average graces in the D range during the ^me period. - 

A review of attendance- rates at the Academy indicated that separated 
students were also lower than remaining students in- this ^category . The 
separated group evidenced average -attendance rates'' of 81%, 82%, ^and 81% for 
the first, second and third quarters respectively." Similar figures for-, 
the remaining group were B6%, 88% and 85%. ' . ' ^ , •• 

In summary the -students who separated from' the Academy during the ' 

tirst project year (n=30) exhibited some similarities and some differences 

f 

when compared with the students who remained in Che Academy program (n=78). 

The- two groups were similar with regard to. sex group membership and types ' 

of sending school, but different in racial* composition . Students who left 

■ " • • ,' 
the Academy were consistently lower than students who remained enrolled on 
/ 

all available performance measures including, sending school- attendance and 
grades, IQ scores, basic skills scores -and Academy attendance and grades." 
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Another point of interest was the cost of ^^perating "the Academy 
prograj||^ In that an effective ^ost tracking, system was not availabl^e 
in FY 1973, sophisticated cost analyses could not be undertaken. The 
data- presented below are based on the project's financial statement as^ j 
of the end of the contract year. This statement contained a description 
of all project expenditures and encumberances as of August 31, 1973. 
The contract was for a 14.5 month period. Operational costs were deter- 
mined for Che .twelve month period beginning September 1, 1972 and ending 
August 31, 1973. These costs were allocated over major operational 
component areas as follows: 

, 1. Administration of oPrograra Operations $ 137,158 

2. General Education * 117,395 ' ' * 

3. Explorative Education 99,902 

4. Specialized Education * 37,146 

5. Guidance and Counseling ' 7^^527^ 

TOTAL ; $ 471,128 

It was not possible to detail costs by task or the learning activity 
classifications used elsewhere in this report. Broadly speaking. General 
Education includes t^ose activities described under Instructional Systems; 
Explorative-Education ihcludes Career Exploration and Life Skills 
Exploration; Specialized Education includes Career Specialisation and 
Life Skills Specialization. ' . 

Several staff perceived problems were indicated by results from the 
Staff Evaluation Reports. Some of these" concerned students, e.g. lack 
of motivation, lateness for class, .attendance problems. But the most 
serious problem was a polarization which seemed to occur between the ^• 

\ 



operational staff and the developmental staff (including management). 
This was reflected in complaints centering on poor. staff relatio^^^hiST 
unclear policy implementation, lack of authority of opefational staff 
and Wk of adminis'trative support., (The syndxsome was seen from the 
operational staff point of view since they were the respondents to the 
evaluation questionnaire.) It may have llso been reflected in the 35% 
rate of §taff turnover during the year, although dat^ on expeqted rates 
for this type ot project are not available. In ajiy case, there were 
apparent staff ^dissatisf actions .. Many operational staff members felt 
that critical .decisions were being made by administrators and developers' , 
who didn't have extensive first-hand 'experience with the students ,c,and . ' 
, that they (operational staff) weren't being adequately supported in the 
fulfillment of their responsibilities. On the other hand, although formal 
data were not ^thered, it seemed that frjany developmental staff felt that 
work on pressing operational problems hindered progress in their develop-' 
mental mission. This polarity of perceptions seems to originate in the 
fact that the prp^lct had discrete operational -and developmental cnais ' 
and staff with ^ prevailin^rationale inte^giatitig the two. It was ' 
fostered by a ^physical separation of the operational staff at the Academy 
site and the d^velopmentaj. staff at the Research for Better Schools officU. 
It was intensified by the' demanding .tjlme schedule for operationalization,^ 
which did not permit testing and refinement of systems b'efore implementation. 
The pressjire for "getting things done" often did not allow time for t(?e * 
internal communication^ and soliciting of staff members ' i'nput^ which 
foster an atmosphere of unity and coordination <)| purpose. So^ of the 
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ingredlettts, -in this|situation ^ simply ftcts of f e'derany-/ifnded ' • 
experimental ?toj,^c|aife'; ^these require *an -a^conunodaaon' of the 7nVf;;^LilI' . 
to the circumsttnc^s. but . the circumstances must be mad;.-cle^t' and held' \ ^ 
in awarenes,. Other ingredients are^ inDerlialr. structureXnd cbmmunrcations c 
system's; these W,,be developed 4 ra»4 possible to diminish fte ^ ' ' 



» » 



problem by informing staff and eh^uraging their input. • ' ■ ' 

As mentioned ^^^bove. the utilization of evaluation , "results was no't 
systematically documented during the first- project' ylar. . The' Ivf l^ation ^ ' 
results were,discuss'ed -in the weekly Project Cabinet Me^ things and'' *ac.t ion . 
was often initiated'^ as a consequence. Since^the evaluabioft effort- 
involved much basic|locumenta'tion^activity evaluation lu't^^ut was Vlso ' 
used in pr-oject mon|toring a«d dercripfio,. Initially all .evaluati'on* 
reports^were disseminated to- ail s'taff,, .During' the year-, in recognition 
of the reports; tect^lcal'Lture; abstrLcta were prepared ,f«.;l^ene|al . 
* dissemination- and re^^orts were available by ^request. Sin^e,; the -^ifer^cts 
contained enough information f or Ij^ral knowledge p-urpot,os, but not 
enough for use in program 'develo^V^= was thc^ght that .the P.at'^^H 
of requests might indicate something about the actual -utilization 'of ' ; 
Valuation results staff. For the'l6 reports disseminated in thit 
manner an average ofSetyeen 4 and 5 copied of each ruir re{)or8 we«- 
■requested (excluding* the 'copies which automatically went to the^Plro^ct 
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Director). The itumb^er requested, ranged . from' 1 to 8 .'r'- Regarding th^ 12 
reports which covered a specific comnonent are'aNhv 8 inkance^ the 
.person(s) responsible for' that area requested; -reUres . C.' Each staff member 
- requested an averagl»of between 3 andH^irepoHsj 'with 'the .\ange ' v ' . A /' 
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extending from 0 to 12. This would' suggest that the evaluation results \ 
were used by staff In som aanner, hut there Is a' loi'of room for ^ - 
lH)rove«ent. Both the tailoring of reports for staff utlli«^Clo&-find the 
documentation of Impact remain in need of ^attention. 
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Recommendations" ^ ^ . ' ' ' . 

Based on the first year experience it seems clear that employers in 
the Philadelphia area are willing and able to successfully provide the 
learning activities which have teen termed Career Exploration and ' 
Specialization. All of the overt criteria for conducting these educational 
experiences for Academy students (n=i07) have been met. Students manifest 
generally positive attitudes toward them, and seem to be learning in'the 
process. The exact nature and extent of .the knowledge conveyed or 
developmen« fostered can only be surmised at present, and remain issues 
for future investigation. It is also clear that the remaining elements 
necessary for a comprehensive educational prog^^can be constructed 
using resources drawn f rom-^gencies and individuals in the community, 
as well, as staff from. the Career Education Project and Research fo7 
Better Sphools. It appears that such a program can affect traditional 
achievement areas. as well as the public schools, while presenting 
^e^'^^fng experiences which are not available in the public- scftSSls. The -7 
answer to the very basic qu'estion, "Can it be done?", is thus affirmative. 

The maximum feasible involvement of 'employers has not yet been, 
tested. The first year experience has led to the conclusion that 
employer responsibility for the total educational program 'might be 
possible experlm'entally, but would haVe limited practical applicability. 
Hence future planning includes the public schools as a cooperating agency, 
while the employer base i» broadened to include direct responsibility 
for all career instruction and guidance activities. The test of these " 
conditions, with only' Basic Skillst instruction conducted by project 
staff, moves significantly closer to the prepaktion of a replicable model. 



The dutcomes will ibe a \at9r^detenn.inW of future direction. 
; /Analysis of the firs^yye^r has also demonstrated that subgroups of . 

• students, ks defined by demyraphic aidv^background'characteristics; .did^, 

• not benefit differentially fiom the pro^^m. ' There is thus 'no bks*^ for ^ 
delimiting a target population on the grinds of expected effect. \ 

, . Several-. .Specific problems or issues" suggested by the evaluation effok 

; •'during the first year have resulted in recommendations aimed-^at improving 
; particular aspects of the program!* 

\ ^ Studen^_ progress in basic skills may have been relatively 'adequate. 

J but it was certainly not exemplary. Many .employers and other instructors 
J commented t^at they could have accomplished much more if students' 
: treading andi mathematics levels had been closer to their expectations 

• Activities ^iimed, at enhancing these skilis should be concentrated on " 

and -the; relatively poorer performance in, mathematTcs should be investigated 
as an indication that more' emphasis^ is needed in^this area. 

The ^W9rld of Work Seminar seemed to be unsuccessful regardless of 
_the progr^ alterations which were attempted. Another means of presenting . 
tje^t^rials and^interaction forumJLnvolved^ should be developed. 1 ' 
, . The.^ Physical Acta,vities Program sounded good but fell on many deaf , 
Wars. It\^.was planned^and executed well, but^s^udents simply didn't 



show mucH:interest. If this area is to remain^ priority, it will have 
to be e'xte-nsively reworked to aslurf meeting stated criterial-' If its 
; relativAyipbr:tance is seen .as nit iustifying^he invesCmentl then" thd 

- ■ V./. . « • . / J r - V-i " 

^ perfon»tlce'.;.crlter.ia should be adjured." : * ' f 

i ' f , ' , I - /. . * ■ 

... The. experiences provided by' thsj^Workshops "dn'. Living seeined to be 

■ ' '4- "'•'Ir' ''''' ' • ' 

^ of value to stud^^ts; whereas, the gj,iacifement (.G;:,jup Counseling)! was 

. ■ ' ■ -'ill ■ . j:';; I • 



.4ess enthusiascic:^lly received. A way- of puling this^aR for students 
should considerefi^v ' . " . , 

. , An integratedl^^nd coordinated approach to records apd documentation 
systems needs to be'Jdeveloped and implemented. Direction should come 
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from- a 'pingle source;^, Since the evaluation staff haS' to deal with the 

' V A- . *■ 

results; and since tH^y have some expertise in the area, ft is suggested 

that this. 'become an evaluation, function. ^ 

A. more sophisCXTac'ed approach to cost accounting needs to be 

developed implemented as a precurspr to cost-benefit studies. This " 
system ^si;>e comprehe^ive and capable of assigning co5t:s to functions. . ' 
The cost.tr^^kingWst^^^hould be closely tied in with the evaluation ' 
. process, wher'e'cost is iatiUnput variable, and th^ specifications process, 
where cost is' difaportanf.f Jr projections. 

The p|:o.lj^em of staff |actionaliza^ion is an important and pficult 
■ one Alth«jugh .the reorganization intd teams having both operational and 
develWntkl iresponsibiXit|es may help, the' separateness of the Chamber ^ 
of Coi^erde staff and thet^ademy functions in their domain may hinder 
communications and inJ.ormi4:lon dissemination. A comprehensive orientation 
• of new stajfj, and^. perhaps ipjd staff as well, should be conducted, to . K 

assure an [inderstaiiding oC' pLrpose and direction. Mechanisms for the * - ^ 

, - ij •-: ! \\\\ 

^ .. inte?actio^,;of Chamber st^f ahd Research for Better Schools staff should " 

be-e^'tablihed.::.Iji.rtic^^^^ should be accorded the team meetings, 

^ ■ wher^ inf^rjiatloA-lfrom th# Project Cabinet may be discussed and staff " ^ 

.in^u^ .soukt. f ;!: , l/ - • . . 

U J^® ^fnfl*ib'l?ic congViMs the mokitoring of ej^luation results 
.-umikatlhti, .The sysrteih.^|,£,abstracting reports for general dissemination 
iHl JBlff fh^: ■ be written t *7^ 

FRir 0-^'l't''l 'tT; '-'hi'} - ^ 
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areas, and they will be written to meet the developmental needs in the 
area concerned. An effort will be made to'define the problems evidenced 
and suggest needed changes. " Technica^discussions will be minin,i.ed in 
the formative reporting and confined mainly to summative documents. 
A follow-up record on each" report will include actions taken as a 
consequence of the results' presented and developers' suggestions, for 
furthpr analysis or ir;vestigat ion. 
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